Reading from Zip archives in Ruby
Tomasz Wegrzanowski
Posted on November 5, 2022
The war isn't going anywhere for now, so every couple of days I have to do the following steps to update Russian losses tracker:
- download zip from Kaggle
- unzip it with unall utility
- run
update_csvscript - verify that data looks right with
git diff, as occasionally there's a typo which makes losses go backwards (always corrected on next update) - delete
archivefolder
The part that I'd like to get rid of is unzipping. We know exactly the path inside the zip, so why bother?
Old script
The script is very straightforward, the methods updated_equipment and updated_personnel will need replacing to get data from a zip.
#!/usr/bin/env ruby
require "pathname"
class UpdateCSV
def initialize(archive_path)
@archive_path = Pathname(archive_path)
end
def updated_equipment
@updated_equipment ||= (@archive_path + "russia_losses_equipment.csv").read
end
def updated_personnel
@updated_personnel ||= (@archive_path + "russia_losses_personnel.csv").read
end
def csv_files
@csv_files ||= `git ls`.lines.map(&:chomp).grep(/\.csv\z/)
end
def call
csv_files.each do |path|
case path
when /russia_losses_equipment/
Pathname(path).write(updated_equipment)
when /russia_losses_personnel/
Pathname(path).write(updated_personnel)
else
puts "Unknown CSV file: #{path}"
end
end
end
end
unless ARGV[0]
STDERR.puts "Usage: #{$0} path_to_updated_archive"
exit 1
end
UpdateCSV.new(ARGV[0]).call
Gem rubyzip vs zip
There's a bit of a problem with Zip for Ruby, as there's two gems - rubyzip and zip. rubyzip is the correct one, zip is an obsolete fork that somehow got a better name.
To use either of them, you need to do require "zip" - which will use whichever one is installed. This is a leftover mess from early days of RubyGems, and such things don't really happen anymore. Just don't gem install zip, and you'll be good.
Abstract archive handling
There's going to be some shared code between updated_equipment and updated_personenel, so let's move it into a new method read_file:
def updated_equipment
@updated_equipment ||= read_file("russia_losses_equipment.csv")
end
def updated_personnel
@updated_personnel ||= read_file("russia_losses_personnel.csv")
end
Read file from either directory or archive
And now the read_file method:
def read_file(path)
if @archive_path.directory?
(@archive_path + path).read
else
Zip::File.open(@archive_path).read(path)
end
end
There are a lot more things we could be doing with zips, but just reading is all we need.
And that's it! It saves me one of the steps.
Full code
Here's the full script:
#!/usr/bin/env ruby
require "pathname"
require "zip"
class UpdateCSV
def initialize(archive_path)
@archive_path = Pathname(archive_path)
end
def read_file(path)
if @archive_path.directory?
(@archive_path + path).read
else
Zip::File.open(@archive_path).read(path)
end
end
def updated_equipment
@updated_equipment ||= read_file("russia_losses_equipment.csv")
end
def updated_personnel
@updated_personnel ||= read_file("russia_losses_personnel.csv")
end
def csv_files
@csv_files ||= `git ls`.lines.map(&:chomp).grep(/\.csv\z/)
end
def call
csv_files.each do |path|
case path
when /russia_losses_equipment/
Pathname(path).write(updated_equipment)
when /russia_losses_personnel/
Pathname(path).write(updated_personnel)
else
puts "Unknown CSV file: #{path}"
end
end
end
end
unless ARGV[0]
STDERR.puts "Usage: #{$0} path_to_updated_archive"
exit 1
end
UpdateCSV.new(ARGV[0]).call
Posted on November 5, 2022
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
November 30, 2024