add csvjson

Author: geraldb

README.md

@@ -5,6 +5,7 @@ Gem Family
 
 [**csvreader**](csvreader) - read tabular data in the comma-separated values (csv) format the right way (uses best practices out-of-the-box with zero-configuration)
 
+[csvjson](csvjson) - read tabular data in the CSV <3 JSON format, that is, comma-separated values CSV (line-by-line) records with javascript object notation (JSON) encoding rules
 
 
 

csvjson/.gitignore

@@ -0,0 +1,12 @@
+#######################
+# ignore ruby rake generated folders
+
+/pkg/
+/doc/
+
+
+################
+# ignore (top-level) datapackage folders
+
+/pack/
+/.pack/

csvjson/HISTORY.md

@@ -0,0 +1,3 @@
+### 0.0.1 / 2018-10-14
+
+* Everything is new. First release

csvjson/Manifest.txt

@@ -0,0 +1,13 @@
+HISTORY.md
+LICENSE.md
+Manifest.txt
+README.md
+Rakefile
+datasets/hello.json.csv
+datasets/hello11.json.csv
+lib/csvjson.rb
+lib/csvjson/parser.rb
+lib/csvjson/version.rb
+test/helper.rb
+test/test_parser.rb
+test/test_parser_misc.rb

csvjson/README.md

@@ -0,0 +1,156 @@
+# CSV <3 JSON Parser / Reader
+
+csvjson library / gem - read tabular data in the CSV <3 JSON format, that is, comma-separated values CSV (line-by-line) records with javascript object notation (JSON) encoding rules
+
+* home  :: [github.com/csvreader/csvjson](https://github.com/csvreader/csvjson)
+* bugs  :: [github.com/csvreader/csvjson/issues](https://github.com/csvreader/csvjson/issues)
+* gem   :: [rubygems.org/gems/csvjson](https://rubygems.org/gems/csvjson)
+* rdoc  :: [rubydoc.info/gems/csvjson](http://rubydoc.info/gems/csvjson)
+* forum :: [wwwmake](http://groups.google.com/group/wwwmake)
+
+
+
+## What's CSV <3 JSON?
+
+CSV <3 JSON is a Comma-Separated Values (CSV)
+variant / format / dialect
+where the line-by-line records follow the
+JavaScript Object Notation (JSON) encoding rules.
+It's a modern (simple) tabular data format that
+includes arrays, numbers, booleans, nulls, nested structures, comments and more.
+Example:
+
+
+```
+# "Vanilla" CSV <3 JSON
+
+1,"John","12 Totem Rd. Aspen",true
+2,"Bob",null,false
+3,"Sue","Bigsby, 345 Carnival, WA 23009",false
+```
+
+or
+
+```
+# CSV <3 JSON with array values
+
+1,"directions",["north","south","east","west"]
+2,"colors",["red","green","blue"]
+3,"drinks",["soda","water","tea","coffe"]
+4,"spells",[]
+```
+
+For more see the [official CSV <3 JSON Format documentation »](https://github.com/csvspecs/csv-json)
+
+
+
+## Usage
+
+``` ruby
+txt <<=TXT
+# "Vanilla" CSV <3 JSON
+
+1,"John","12 Totem Rd. Aspen",true
+2,"Bob",null,false
+3,"Sue","Bigsby, 345 Carnival, WA 23009",false
+TXT
+
+records = CsvJson.parse( txt )     ## or CSV_JSON.parse or CSVJSON.parse
+pp records
+# =>  [[1,"John","12 Totem Rd. Aspen",true],
+#      [2,"Bob",nil,false],
+#      [3,"Sue","Bigsby, 345 Carnival, WA 23009",false]]
+
+# -or-
+
+records = CsvJson.read( "values.json.csv" )   ## or CSV_JSON.read or CSVJSON.read
+pp records
+# =>  [[1,"John","12 Totem Rd. Aspen",true],
+#      [2,"Bob",nil,false],
+#      [3,"Sue","Bigsby, 345 Carnival, WA 23009",false]]
+
+# -or-
+
+CsvJson.foreach( "values.json.csv" ) do |rec|   ## or CSV_JSON.foreach or CSVJSON.foreach
+  pp rec
+end
+# => [1,"John","12 Totem Rd. Aspen",true]
+# => [2,"Bob",nil,false]
+# => [3,"Sue","Bigsby, 345 Carnival, WA 23009",false]
+```
+
+
+
+### What about Enumerable?
+
+Yes, the reader / parser includes `Enumerable` and runs on `each`.
+Use `new` or `open` without a block
+to get the enumerator (iterator).
+Example:
+
+
+``` ruby
+csv = CsvJson.new( "1,2,3" )   ## or CSV_JSON.new or CSVJSON.new
+it  = csv.to_enum
+pp it.next  
+# => [1,2,3]
+
+# -or-
+
+csv = CsvJson.open( "values.json.csv" )  ## or CSV_JSON.open or CSVJSON.open
+it  = csv.to_enum
+pp it.next
+# => [1,"John","12 Totem Rd. Aspen",true]
+pp it.next
+# => [2,"Bob",nil,false]
+```
+
+
+
+### What about headers?
+
+Yes, you can. Use the `CsvHash`
+from the csvreader library / gem
+if the first line is a header (or if missing pass in the headers
+as an array) and you want your records as hashes instead of arrays of strings.
+Example:
+
+``` ruby
+txt <<=TXT
+"id","name","address","regular"
+1,"John","12 Totem Rd. Aspen",true
+2,"Bob",null,false
+3,"Sue","Bigsby, 345 Carnival, WA 23009",false
+TXT
+
+records = CsvHash.json.parse( txt )
+pp records
+
+# => [{"id":      1,
+#      "name":    "John",
+#      "address": "12 Totem Rd. Aspen",
+#      "regular": true},
+#     {"id":      2,
+#      "name":    "Bob",
+#      "address": null,
+#      "regular": false},
+#    ... ]
+```
+
+For more see the [official CsvHash documentation in the csvreader library / gem »](https://github.com/csvreader/csvreader)
+
+
+
+
+
+
+## License
+
+The `csvjson` scripts are dedicated to the public domain.
+Use it as you please with no restrictions whatsoever.
+
+
+## Questions? Comments?
+
+Send them along to the [wwwmake forum](http://groups.google.com/group/wwwmake).
+Thanks!

csvjson/Rakefile

@@ -0,0 +1,29 @@
+require 'hoe'
+require './lib/csvjson/version.rb'
+
+Hoe.spec 'csvjson' do
+
+  self.version = CsvJson::VERSION
+
+  self.summary = "csvjson - read tabular data in the CSV <3 JSON format, that is, comma-separated values CSV (line-by-line) records with javascript object notation (JSON) encoding rules"
+  self.description = summary
+
+  self.urls    = ['https://github.com/csvreader/csvjson']
+
+  self.author  = 'Gerald Bauer'
+  self.email   = '[email protected]'
+
+  # switch extension to .markdown for gihub formatting
+  self.readme_file  = 'README.md'
+  self.history_file = 'HISTORY.md'
+
+  self.extra_deps = [
+  ]
+
+  self.licenses = ['Public Domain']
+
+  self.spec_extras = {
+    required_ruby_version: '>= 2.2.2'
+  }
+
+end

csvjson/datasets/hello.json.csv

@@ -0,0 +1,3 @@
+1,"John","12 Totem Rd. Aspen",true
+2,"Bob",null,false
+3,"Sue","Bigsby, 345 Carnival, WA 23009",false

csvjson/datasets/hello11.json.csv

@@ -0,0 +1,5 @@
+# hello world
+
+1, "John", "12 Totem Rd. Aspen",             true
+2, "Bob",  null,                             false
+3, "Sue",  "Bigsby, 345 Carnival, WA 23009", false

csvjson/lib/csvjson.rb

@@ -0,0 +1,22 @@
+# encoding: utf-8
+
+require 'pp'
+require 'json'
+require 'logger'
+
+
+## our own code
+##   todo/check: use require_relative - why? why not?
+require 'csvjson/version'    # note: let version always go first
+require 'csvjson/parser'
+
+
+## add some "alternative" shortcut aliases
+CSV_JSON = CsvJson
+CSVJSON  = CsvJson
+CSVJ     = CsvJson
+CsvJ     = CsvJson
+
+
+# say hello
+puts CsvJson.banner    if $DEBUG || (defined?($RUBYCOCO_DEBUG) && $RUBYCOCO_DEBUG)

csvjson/lib/csvjson/parser.rb

@@ -0,0 +1,131 @@
+# encoding: utf-8
+
+
+class CsvJson
+
+###################################
+## add simple logger with debug flag/switch
+#
+#  use Parser.debug = true   # to turn on
+#
+#  todo/fix: use logutils instead of std logger - why? why not?
+
+def self.build_logger()
+  l = Logger.new( STDOUT )
+  l.level = :info    ## set to :info on start; note: is 0 (debug) by default
+  l
+end
+def self.logger() @@logger ||= build_logger; end
+def logger()  self.class.logger; end
+
+
+
+
+def self.open( path, mode=nil, &block )   ## rename path to filename or name - why? why not?
+
+    ## note: default mode (if nil/not passed in) to 'r:bom|utf-8'
+    f = File.open( path, mode ? mode : 'r:bom|utf-8' )
+    csv = new( f )
+
+    # handle blocks like Ruby's open()
+    if block_given?
+      begin
+        block.call( csv )
+      ensure
+        csv.close
+      end
+    else
+      csv
+    end
+end # method self.open
+
+
+def self.read( path )
+    open( path ) { |csv| csv.read }
+end
+
+
+def self.foreach( path, &block )
+  csv = open( path )
+
+  if block_given?
+    begin
+      csv.each( &block )
+    ensure
+      csv.close
+    end
+  else
+    csv.to_enum    ## note: caller (responsible) must close file!!!
+    ## remove version without block given - why? why not?
+    ## use Csv.open().to_enum  or Csv.open().each
+    ##   or Csv.new( File.new() ).to_enum or Csv.new( File.new() ).each ???
+  end
+end # method self.foreach
+
+
+def self.parse( data, &block )
+  csv = new( data )
+
+  if block_given?
+    csv.each( &block )  ## note: caller (responsible) must close file!!! - add autoclose - why? why not?
+  else  # slurp contents, if no block is given
+    csv.read            ## note: caller (responsible) must close file!!! - add autoclose - why? why not?
+  end
+end # method self.parse
+
+
+
+def initialize( data )
+  if data.is_a?( String )
+    @input = data   # note: just needs each for each_line
+  else  ## assume io
+    @input = data
+  end
+end
+
+
+
+include Enumerable
+
+def each( &block )
+  if block_given?
+    @input.each_line do |line|
+
+      logger.debug  "line:"             if logger.debug?
+      logger.debug line.pretty_inspect  if logger.debug?
+
+
+      ##  note: chomp('') if is an empty string,
+      ##    it will remove all trailing newlines from the string.
+      ##    use line.sub(/[\n\r]*$/, '') or similar instead - why? why not?
+      line = line.chomp( '' )
+      line = line.strip         ## strip leading and trailing whitespaces (space/tab) too
+      logger.debug line.pretty_inspect    if logger.debug?
+
+      if line.empty?             ## skip blank lines
+        logger.debug "skip blank line"    if logger.debug?
+        next
+      end
+
+      if line.start_with?( "#" )  ## skip comment lines
+        logger.debug "skip comment line"   if logger.debug?
+        next
+      end
+
+      ## note: auto-wrap in array e.g. with []
+      json = JSON.parse( "[#{line}]" )
+      logger.debug json.pretty_inspect   if logger.debug?
+      block.call( json )
+    end
+  else
+     to_enum
+  end
+end # method each
+
+def read() to_a; end # method read
+
+def close
+  @input.close   if @input.respond_to?(:close)   ## note: string needs no close
+end
+
+end # class CsvJson