Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Geocode addresses on import #326

Merged
merged 27 commits into from
Jul 8, 2024
Merged
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
f99d6b5
Adding CSV importer with basic import and validation functions, and r…
ut Jul 17, 2023
35b783f
Rubocop. Adding frozen_string_literal
ut Jul 19, 2023
15eed71
Rubocop/2
ut Jul 19, 2023
1dded04
Adding CSV header validation, define required headers and fields, add…
ut Jul 21, 2023
53bc39a
Rubocop'd
ut Jul 21, 2023
f5c7f2f
Sanitize/strip HTML from inputs
ut Jul 22, 2023
6e920a8
Add allowed_fields routine and create all allowed fields with Place.new
ut Jul 22, 2023
5bd60c2
Rspec for nonallowed fields, improve sanitzer
ut Jul 24, 2023
d9cdd17
Adding controller logic, form template
ut Jul 29, 2023
d34daa4
Rspec for controller and routing
ut Jul 29, 2023
534ca12
Import preview
ut Jul 31, 2023
f28f175
get import file by param
ut Jul 31, 2023
edaf4df
Swith to session based datatransfer between steps
ut Jul 31, 2023
2ec6de0
Merge branch 'main' into ut/108-csv-import
ut Aug 21, 2023
7703693
Merge branch 'main' into ut/108-csv-import
ut Aug 22, 2023
f2eda21
Show data errors w/error types and AR messages
ut Aug 22, 2023
3e1cb8e
Improve import preview output/UI. Adapt rspecs (tbc)
ut Aug 23, 2023
181f208
Minor improvements, better comments,...
ut Aug 23, 2023
456569e
Fix/complete RSpecs
ut Aug 23, 2023
9580d76
Merge branch 'main' into ut/108-csv-import
ut Aug 23, 2023
2b42060
Link import, improve descriptions
ut Aug 25, 2023
7b97f62
Rubocop settings
ut Aug 26, 2023
e25e00c
Enable overwrite feature. Capture wrong/invalid CSV format error
ut Aug 26, 2023
2411af2
Fix missing keyword param in controller
ut Aug 26, 2023
424e75d
Merge branch 'main' into ut/108-csv-import
ut Sep 11, 2023
c6daff1
Merge branch 'main' into ut/108-csv-import
ut Jul 8, 2024
708a60b
Merged main manually
ut Jul 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Rubocop. Adding frozen_string_literal
ut committed Jul 19, 2023
commit 35b783f8c0171be30fd6f227b42837f82333bb21
17 changes: 9 additions & 8 deletions lib/imports/csv_importer.rb
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# frozen_string_literal: true

require 'csv'

class Imports::CsvImporter
attr_reader :invalid_rows


def initialize(file,layer_id)
def initialize(file, layer_id)
@file = file
@invalid_rows = []
@layer = Layer.find(layer_id)
@@ -44,13 +45,13 @@ def valid_row?(row)
def process_valid_rows
CSV.foreach(@file.path, headers: true) do |row|
title = sanitize(row['title'])
unless @existing_titles.include?(title)
place = Place.new(title: title, teaser: sanitize(row['teaser']), layer_id: @layer.id)
if @existing_titles.include?(title)
Rails.logger.error("Place already exists! #{title}")
puts 'Place already exists'
else
place = Place.new(title: title, teaser: sanitize(row['teaser']), layer_id: @layer.id)
place.save!
@existing_titles << title
else
Rails.logger.error("Place already exists! #{title}")
puts "Place already exists"
end
end
end
14 changes: 7 additions & 7 deletions spec/lib/csv_importer_spec.rb
Original file line number Diff line number Diff line change
@@ -10,12 +10,12 @@

context 'with valid CSV' do
it 'creates new Place records from valid rows' do
importer = Imports::CsvImporter.new(file,layer.id)
importer = Imports::CsvImporter.new(file, layer.id)

expect {
# Assuming the CSV file has 3 valid rows
expect do
importer.import
}.to change(Place, :count).by(3) # Assuming the CSV file has 3 valid rows

end.to change(Place, :count).by(3)
# Additional assertions if needed
expect(Place.pluck(:title)).to contain_exactly('Place 1', 'Place 2', 'Place 3')
end
@@ -25,11 +25,11 @@
it 'handles invalid rows and does not create Place records' do
invalid_file = Rack::Test::UploadedFile.new('spec/support/files/places_invalid.csv', 'text/csv')

importer = Imports::CsvImporter.new(invalid_file,layer.id)
importer = Imports::CsvImporter.new(invalid_file, layer.id)

expect {
expect do
importer.import
}.not_to change(Place, :count)
end.not_to change(Place, :count)

expect(importer.invalid_rows.count).to eq(1)
end