summaryrefslogtreecommitdiffstats
path: root/tcllib/modules/csv
diff options
context:
space:
mode:
authorWilliam Joye <wjoye@cfa.harvard.edu>2016-10-27 19:39:39 (GMT)
committerWilliam Joye <wjoye@cfa.harvard.edu>2016-10-27 19:39:39 (GMT)
commitea28451286d3ea4a772fa174483f9a7a66bb1ab3 (patch)
tree6ee9d8a7848333a7ceeee3b13d492e40225f8b86 /tcllib/modules/csv
parentb5ca09bae0d6a1edce939eea03594dd56383f2c8 (diff)
parent7c621da28f07e449ad90c387344f07a453927569 (diff)
downloadblt-ea28451286d3ea4a772fa174483f9a7a66bb1ab3.zip
blt-ea28451286d3ea4a772fa174483f9a7a66bb1ab3.tar.gz
blt-ea28451286d3ea4a772fa174483f9a7a66bb1ab3.tar.bz2
Merge commit '7c621da28f07e449ad90c387344f07a453927569' as 'tcllib'
Diffstat (limited to 'tcllib/modules/csv')
-rw-r--r--tcllib/modules/csv/2926387.csv4
-rw-r--r--tcllib/modules/csv/ChangeLog339
-rw-r--r--tcllib/modules/csv/csv.bench45
-rw-r--r--tcllib/modules/csv/csv.man247
-rw-r--r--tcllib/modules/csv/csv.pcx144
-rw-r--r--tcllib/modules/csv/csv.tcl789
-rw-r--r--tcllib/modules/csv/csv.test998
-rw-r--r--tcllib/modules/csv/eval.csv6
-rw-r--r--tcllib/modules/csv/mem_debug_bench.csv251
-rw-r--r--tcllib/modules/csv/mem_debug_bench_a.csv256
-rw-r--r--tcllib/modules/csv/pkgIndex.tcl2
11 files changed, 3081 insertions, 0 deletions
diff --git a/tcllib/modules/csv/2926387.csv b/tcllib/modules/csv/2926387.csv
new file mode 100644
index 0000000..95d0a9c
--- /dev/null
+++ b/tcllib/modules/csv/2926387.csv
@@ -0,0 +1,4 @@
+a,b,c
+d,"e,
+e",f
+
diff --git a/tcllib/modules/csv/ChangeLog b/tcllib/modules/csv/ChangeLog
new file mode 100644
index 0000000..19fc8e1
--- /dev/null
+++ b/tcllib/modules/csv/ChangeLog
@@ -0,0 +1,339 @@
+2013-02-01 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.15 ========================
+ *
+
+2013-01-08 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.tcl: [Bug 3575707]: Actually a feature change, the commands
+ * csv.test: join, joinlist, and joinmatrix are extended with a flag
+ * csv.pcx: argument to force use of the delimiter/quoting character,
+ * csv.man: regardless of need. Original patch by Pietro Cerutti
+ * pkgIndex.tcl: <gahr@users.sourceforge.net>. Version bumped to 0.8
+
+2011-12-13 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.14 ========================
+ *
+
+2011-11-22 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.tcl: [Bug 1724818]: Applied the patch supplied by Jeremy
+ * csv.man: Cowgar <jeremy@cowgar.com> fixing the issue. Bumped
+ * csv.test: version to 0.7.3. Extended testsuite.
+ * pkgIndex.tcl:
+
+2011-04-11 Andreas Kupries <andreask@activestate.com>
+
+ * csv.man: [Bug 3281791]: Followup to fix for [Bug 3061815], fixed
+ forgotten change in the text after the examples. Thanks to
+ <guardus@users.sourceforge.net>.
+
+2011-01-24 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.13 ========================
+ *
+
+2010-09-08 Andreas Kupries <andreask@activestate.com>
+
+ * csv.man: [Bug 3061815]: Fixed a mixup in the examples which
+ matched regular output to alternate format and vice versa. Thanks
+ to Harald Oehlmann <oehhar@users.sourceforge.net>.
+
+2010-01-19 Andreas Kupries <andreask@activestate.com>
+
+ * csv.tcl (::csv::read2queue): [Bug 2926387]: Fix use of wrong
+ * csv.test: variable when handling multi-line fields reported by
+ * csv.man: Jeff Rogers <dvrsn@users.sourceforge.net>. Extended
+ * pkgIndex.tcl: testsuite. Bumped version to 0.7.2.
+ * 2926387.csv: <New file>, for the new tests.
+
+2009-12-07 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.12 ========================
+ *
+
+2009-09-17 Andreas Kupries <andreask@activestate.com>
+
+ * csv.man: [Bug 2860843]. Fixed two documentation typos reported
+ by Larry Virden <lvirden@users.sourceforge.net>
+
+2008-12-12 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.11.1 ========================
+ *
+
+2008-10-16 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.11 ========================
+ *
+
+2008-10-02 Andreas Kupries <andreask@activestate.com>
+
+ * csv.tcl: Fixed [SF Bug 2123513]. Added protections against
+ * csv.man: malformed separator characters (empty or string) to the
+ * csv.test: read2 and split2 commands. Extended test suite to
+ * pkgIndex.tcl: cover these cases. Bumped the package version to
+ 0.7.1.
+
+2008-06-14 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.pcx: New file. Syntax definitions for the public commands of
+ the csv package.
+
+2007-09-12 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.10 ========================
+ *
+
+2007-03-21 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.man: Fixed all warnings due to use of now deprecated
+ commands. Added a section about how to give feedback.
+
+2006-10-03 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.9 ========================
+ *
+
+2006-09-19 Andreas Kupries <andreask@activestate.com>
+
+ * csv.man: Bumped version to 0.7.
+ * csv.tcl:
+ * pkgIndex.tcl
+
+2006-06-15 Andreas Kupries <andreask@activestate.com>
+
+ * csv.tcl: Extended csv processing to allow different
+ * csv.test: quoting chars beyond double-quote. Patch origin at [SF
+ * csv.man: Tcllib Patch 1469593]. Needed small fix in
+ join. Extended testsuite, documentation.
+
+2006-01-28 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.test: Fixed use and cleanup of temp. files.
+
+2006-01-22 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.test: More boilerplate simplified via use of test support.
+
+2006-01-21 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.test: Removed some 8.4'isms out of the csv testsuite, the
+ package under test works for 8.3+.
+
+2006-01-19 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.test: Hooked into the new common test support code.
+
+2006-01-16 Andreas Kupries <akupries@shaw.ca>
+
+ * csv.man: New command 'iscomplete' to detect partial csv
+ * csv.tcl: records. Used to enable the read2* commands to handle
+ multi-line csv records. Code provided by Jeff Hobbs, via [SF
+ Tcllib Patch 1407811]. See also the [Tcllib FR 733407].
+
+2005-10-24 Andreas Kupries <andreask@activestate.com>
+
+ * csv.bench: New file. Basic benchmarks for CSV processing.
+
+2005-10-06 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.8 ========================
+ *
+
+2005-09-30 Andreas Kupries <andreask@activestate.com>
+
+ * pkgIndex.tcl: Added command 'csv::joinmatrix', which converts a
+ * csv.man: matrix object into CSV records, one record per
+ * csv.tcl: row. Inspired by [SF Tcllib RFE 1204345] which
+ brought the conversion up, but went a round-about
+ way via a report object.
+
+2005-04-13 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.test: Testsuite package requirements fixed to ensure use of
+ local packages.
+
+2004-10-05 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.7 ========================
+ *
+
+2004-05-23 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.tcl: Updated version number to sync with 1.6.1
+ * csv.man: release.
+ * pkgIndex.tcl:
+
+2004-05-23 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.6.1 ========================
+ *
+
+2004-05-23 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.tcl: Rel. engineering. Updated version number
+ * csv.man: of csv to reflect its changes, to 0.5.1.
+ * pkgIndex.tcl:
+
+2004-05-03 Andreas Kupries <andreask@pliers.activestate.com>
+
+ * csv.tcl (read2matrix): Fixed bogus switch case. Had case "4"
+ twice, second should have been "5". [SF Tcllib Bug 940651].
+
+2004-02-15 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.6 ========================
+ *
+
+2003-11-22 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.man: Extended the explanation for the example to cover the
+ alternate format as well [SF Tcllib RFE 737770].
+
+2003-05-12 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.man: Changed the phrasing for the alternate format a bit,
+ and reworded the text enclosing the example.
+
+2003-05-05 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ *
+ * Released and tagged Tcllib 1.4 ========================
+ *
+
+2003-04-24 Andreas Kupries <andreask@activestate.com>
+
+ * csv.tcl: Bumped version to 0.4. This had been
+ * csv.man: forgotten before.
+ * pkgIndex.tcl:
+
+2003-04-23 Andreas Kupries <andreask@activestate.com>
+
+ * csv.tcl (Split): Rewrote parser for alternate syntax to handle
+ the remaining known bug. Now it passes the testsuite completely.
+
+ * csv.man: Extended to handle a slightly different alternate
+ * cvs.tcl: syntax of CSV files. This takes care of bug
+ * csv.test: [606141].
+
+2003-03-31 Andreas Kupries <andreask@activestate.com>
+
+ * csv.tcl (split): Fixed bug #709123 reported by Jamie Honan
+ <jhonan@users.sourceforge.net>. The separator character is used
+ in regular epxressions, but was not protected against special
+ interpretation by the RE engine.
+
+2003-01-16 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.man: More semantic markup, less visual one.
+
+2002-06-24 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.tcl (csv::split): Fixed bug #565051, found by Tod A. olson
+ <todolson@users.sourceforge.net>. The described bug is actually
+ none, given the definition of the CSV format, but the examples
+ do contain a related bug. Just swap what is seen as ok and
+ bug. Because of this the provided patched code was rejected, and
+ a new patch created. The patched code passes the extended
+ testsuite (see below).
+
+ * csv.test: Extended testsuite regarding the handling of empty
+ fields and quote characters. Part of the investigation into bug
+ #565051.
+
+2002-03-25 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.man: Fixed formatting errors in the doctools manpage.
+
+2002-02-01 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * Version up to 0.3 to differentiate development from the
+ version in the tcllib 1.2 release.
+
+ * mem_debug_bench_a.csv: New file, contains empty lines to test
+ that part of the code. See below.
+ * csv.tcl:
+ * csv.test: Updated code and tests to cover all paths through the
+ code.
+
+2002-01-15 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * Bumped version to 0.2
+
+2001-11-16 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.n: Applied patch #482570 correcting a typo and adding more
+ cross-references (see also, keywords). Patch provided by Larry
+ Virden <lvirden@users.sourceforge.net>.
+
+2001-11-12 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.test:
+ * cvs.n:
+ * csv.tcl (split2matrix, read2matrix): Implemented FR
+ #481023. Added additional expansion behaviours, controlled via
+ an optional argument.
+
+2001-10-14 Jeff Hobbs <jeffh@ActiveState.com>
+
+ * csv.test (csv-1.7):
+ * csv.tcl: Fixed [Bug #469855] where starting "s could not come
+ out right from csv::split.
+ Updated to 0.2
+
+2001-09-28 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.test: Added test to verify that the problem is fixed.
+
+ * csv.tcl (joinlist): Fixed bug [#465210] "::csv::joinlist
+ sepChar handling". The "sepChar" was not propagated to the
+ actual join operation.
+
+2001-09-05 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.tcl: Restricted export list to public API.
+ [456255]. Patch by Hemang Lavana
+ <hemanglavana@users.sourceforge.net>
+
+2001-07-10 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.tcl: Frink 2.2 run, fixed dubious code.
+
+2001-06-21 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.tcl: Fixed dubious code reported by frink and procheck.
+
+2001-06-19 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.n: Fixed nroff trouble.
+
+2001-05-01 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * Committed to CVS head at SF.
+
+2001-04-18 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * csv.tcl: Added more code to read and write CSV formatted data
+ from and to various datastructures (queue, matrix). The basic
+ functionality is now complete.
+
+ * csv.test: Extended the testsuite to cover the new code.
+ * csv.n: Extended the documentation to cover the new code.
+
+2001-04-12 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+ * New module for the processing of CSV lines and files.
diff --git a/tcllib/modules/csv/csv.bench b/tcllib/modules/csv/csv.bench
new file mode 100644
index 0000000..44b21be
--- /dev/null
+++ b/tcllib/modules/csv/csv.bench
@@ -0,0 +1,45 @@
+# -*- tcl -*-
+# Tcl Benchmark File
+#
+# This file contains a number of benchmarks for the 'csv' module.
+# This allow developers to monitor/gauge/track package performance.
+#
+# (c) 2005 Andreas Kupries <andreas_kupries@users.sourceforge.net>
+
+# We need at least version 8.2 for the package and thus the
+# benchmarks.
+
+if {![package vsatisfies [package provide Tcl] 8.3]} {
+ return
+}
+
+# ### ### ### ######### ######### ######### ###########################
+## Setting up the environment ...
+
+package forget csv
+catch {namespace delete ::csv}
+source [file join [file dirname [info script]] csv.tcl]
+
+# ### ### ### ######### ######### ######### ###########################
+## Benchmarks.
+
+foreach n {1 10 100 1000 10000} {
+ bench -desc "CSV join $n" -pre {
+ set list [split [string repeat " " $n] ""]
+ } -body {
+ csv::join $list
+ } -post {
+ unset list
+ }
+
+ bench -desc "CSV split $n" -pre {
+ set str [string repeat , $n]
+ } -body {
+ csv::split $str
+ } -post {
+ unset str
+ }
+}
+
+# ### ### ### ######### ######### ######### ###########################
+## Complete
diff --git a/tcllib/modules/csv/csv.man b/tcllib/modules/csv/csv.man
new file mode 100644
index 0000000..76d3259
--- /dev/null
+++ b/tcllib/modules/csv/csv.man
@@ -0,0 +1,247 @@
+[comment {-*- tcl -*-}]
+[vset VERSION 0.8.1]
+[manpage_begin csv n [vset VERSION]]
+[see_also matrix]
+[see_also queue]
+[keywords csv]
+[keywords matrix]
+[keywords package]
+[keywords queue]
+[keywords tcllib]
+[copyright {2002-2015 Andreas Kupries <andreas_kupries@users.sourceforge.net>}]
+[moddesc {CSV processing}]
+[titledesc {Procedures to handle CSV data.}]
+[category {Text processing}]
+[require Tcl 8.4]
+[require csv [opt [vset VERSION]]]
+[description]
+
+[para]
+
+The [package csv] package provides commands to manipulate information
+in CSV [sectref FORMAT] (CSV = Comma Separated Values).
+
+[section COMMANDS]
+[para]
+
+The following commands are available:
+
+[list_begin definitions]
+
+[call [cmd ::csv::iscomplete] [arg data]]
+
+A predicate checking if the argument [arg data] is a complete csv
+record. The result is a boolean flag indicating the completeness of
+the data. The result is true if the data is complete.
+
+[call [cmd ::csv::join] [arg values] [opt [arg sepChar]] [opt [arg delChar]] [opt [arg delMode]]]
+
+Takes a list of values and returns a string in CSV format containing
+these values. The separator character can be defined by the caller,
+but this is optional. The default is ",". The quoting aka delimiting character can
+be defined by the caller, but this is optional. The default is '"'.
+
+By default the quoting mode [arg delMode] is "auto", surrounding
+values with [arg delChar] only when needed. When set to "always"
+however, values are always surrounded by the [arg delChar] instead.
+
+[call [cmd ::csv::joinlist] [arg values] [opt [arg sepChar]] [opt [arg delChar]] [opt [arg delMode]]]
+
+Takes a list of lists of values and returns a string in CSV format
+containing these values. The separator character can be defined by the
+caller, but this is optional. The default is ",". The quoting character
+can be defined by the caller, but this is optional. The default is '"'.
+
+By default the quoting mode [arg delMode] is "auto", surrounding
+values with [arg delChar] only when needed. When set to "always"
+however, values are always surrounded by the [arg delChar] instead.
+
+Each element of the outer list is considered a record, these are
+separated by newlines in the result. The elements of each record are
+formatted as usual (via [cmd ::csv::join]).
+
+[call [cmd ::csv::joinmatrix] [arg matrix] [opt [arg sepChar]] [opt [arg delChar]] [opt [arg delMode]]]
+
+Takes a [arg matrix] object following the API specified for the
+struct::matrix package and returns a string in CSV format containing
+these values. The separator character can be defined by the caller,
+but this is optional. The default is ",". The quoting character
+can be defined by the caller, but this is optional. The default is
+'"'.
+
+By default the quoting mode [arg delMode] is "auto", surrounding
+values with [arg delChar] only when needed. When set to "always"
+however, values are always surrounded by the [arg delChar] instead.
+
+Each row of the matrix is considered a record, these are
+separated by newlines in the result. The elements of each record are
+formatted as usual (via [cmd ::csv::join]).
+
+[call [cmd ::csv::read2matrix] [opt [option -alternate]] [arg "chan m"] "{[arg sepChar] ,} {[arg expand] none}"]
+
+A wrapper around [cmd ::csv::split2matrix] (see below) reading
+CSV-formatted lines from the specified channel (until EOF) and adding
+them to the given matrix. For an explanation of the [arg expand]
+argument see [cmd ::csv::split2matrix].
+
+[call [cmd ::csv::read2queue] [opt [option -alternate]] [arg "chan q"] "{[arg sepChar] ,}"]
+
+A wrapper around [cmd ::csv::split2queue] (see below) reading
+CSV-formatted lines from the specified channel (until EOF) and adding
+them to the given queue.
+
+[call [cmd ::csv::report] [arg "cmd matrix"] [opt [arg chan]]]
+
+A report command which can be used by the matrix methods
+
+[cmd "format 2string"] and [cmd "format 2chan"]. For the latter this
+command delegates the work to [cmd ::csv::writematrix]. [arg cmd] is
+expected to be either [method printmatrix] or
+
+[method printmatrix2channel]. The channel argument, [arg chan], has
+to be present for the latter and must not be present for the first.
+
+[call [cmd ::csv::split] [opt [option -alternate]] [arg line] [opt [arg sepChar]] [opt [arg delChar]]]
+
+converts a [arg line] in CSV format into a list of the values
+contained in the line. The character used to separate the values from
+each other can be defined by the caller, via [arg sepChar], but this
+is optional. The default is ",". The quoting character can be defined
+by the caller, but this is optional. The default is '"'.
+
+[para]
+
+If the option [option -alternate] is specified a slightly different
+syntax is used to parse the input. This syntax is explained below, in
+the section [sectref FORMAT].
+
+[call [cmd ::csv::split2matrix] [opt [option -alternate]] [arg "m line"] "{[arg sepChar] ,} {[arg expand] none}"]
+
+The same as [cmd ::csv::split], but appends the resulting list as a
+new row to the matrix [arg m], using the method [cmd "add row"]. The
+expansion mode specified via [arg expand] determines how the command
+handles a matrix with less columns than contained in [arg line]. The
+allowed modes are:
+
+[list_begin definitions]
+
+[def [const none]]
+
+This is the default mode. In this mode it is the responsibility of the
+caller to ensure that the matrix has enough columns to contain the
+full line. If there are not enough columns the list of values is
+silently truncated at the end to fit.
+
+[def [const empty]]
+
+In this mode the command expands an empty matrix to hold all columns
+of the specified line, but goes no further. The overall effect is that
+the first of a series of lines determines the number of columns in the
+matrix and all following lines are truncated to that size, as if mode
+[const none] was set.
+
+[def [const auto]]
+
+In this mode the command expands the matrix as needed to hold all
+columns contained in [arg line]. The overall effect is that after
+adding a series of lines the matrix will have enough columns to hold
+all columns of the longest line encountered so far.
+
+[list_end]
+
+[call [cmd ::csv::split2queue] [opt [option -alternate]] [arg "q line"] "{[arg sepChar] ,}"]
+
+The same as [cmd ::csv::split], but appending the resulting list as a
+single item to the queue [arg q], using the method [cmd put].
+
+[call [cmd ::csv::writematrix] [arg "m chan"] [opt [arg sepChar]] [opt [arg delChar]]]
+
+A wrapper around [cmd ::csv::join] taking all rows in the matrix
+[arg m] and writing them CSV formatted into the channel [arg chan].
+
+[call [cmd ::csv::writequeue] [arg "q chan"] [opt [arg sepChar]] [opt [arg delChar]]]
+
+A wrapper around [cmd ::csv::join] taking all items in the queue
+[arg q] (assumes that they are lists) and writing them CSV formatted
+into the channel [arg chan].
+
+[list_end]
+
+[section FORMAT]
+[para]
+
+The format of regular CSV files is specified as
+
+[list_begin enumerated]
+
+[enum]
+Each record of a csv file (comma-separated values, as exported e.g. by
+Excel) is a set of ASCII values separated by ",". For other languages
+it may be ";" however, although this is not important for this case as
+the functions provided here allow any separator character.
+
+[enum]
+If and only if a value contains itself the separator ",", then it (the
+value) has to be put between "". If the value does not contain the
+separator character then quoting is optional.
+
+[enum]
+If a value contains the character ", that character is represented by "".
+
+[enum]
+The output string "" represents the value ". In other words, it is
+assumed that it was created through rule 3, and only this rule,
+i.e. that the value was not quoted.
+
+[list_end]
+[para]
+
+An alternate format definition mainly used by MS products specifies
+that the output string "" is a representation of the empty
+string. In other words, it is assumed that the output was generated
+out of the empty string by quoting it (i.e. rule 2), and not through
+rule 3. This is the only difference between the regular and the
+alternate format.
+
+[para]
+
+The alternate format is activated through specification of the option
+[option -alternate] to the various split commands.
+
+[section EXAMPLE]
+
+Using the regular format the record
+
+[para]
+[example {
+123,"123,521.2","Mary says ""Hello, I am Mary""",""
+}]
+
+[para]
+is parsed into the items
+
+[para]
+[example {
+a) 123
+b) 123,521.2
+c) Mary says "Hello, I am Mary"
+d) "
+}]
+[para]
+
+Using the alternate format the result is
+
+[para]
+[example {
+a) 123
+b) 123,521.2
+c) Mary says "Hello, I am Mary"
+d) (the empty string)
+}]
+
+instead. As can be seen only item (d) is different, now the empty string
+instead of a ".
+
+[vset CATEGORY csv]
+[include ../doctools2base/include/feedback.inc]
+[manpage_end]
diff --git a/tcllib/modules/csv/csv.pcx b/tcllib/modules/csv/csv.pcx
new file mode 100644
index 0000000..fee8344
--- /dev/null
+++ b/tcllib/modules/csv/csv.pcx
@@ -0,0 +1,144 @@
+# -*- tcl -*- csv.pcx
+# Syntax of the commands provided by package csv.
+
+# For use by TclDevKit's static syntax checker.
+# See http://www.activestate.com/solutions/tcl/
+# See http://aspn.activestate.com/ASPN/docs/Tcl_Dev_Kit/4.0/Checker.html#pcx_api
+# for the documentation describing the format of the code contained in this file
+#
+
+package require pcx
+pcx::register csv
+pcx::tcldep 0.7 needs tcl 8.3
+pcx::tcldep 0.8 needs tcl 8.4
+
+namespace eval ::csv {}
+
+#pcx::message FOO {... text ...} type
+#pcx::scan <VERSION> <NAME> <RULE>
+
+pcx::check 0.7 std ::csv::iscomplete \
+ {checkSimpleArgs 1 1 {
+ checkWord
+ }}
+pcx::check 0.7 std ::csv::join \
+ {checkSimpleArgs 1 3 {
+ checkList
+ checkWord
+ checkWord
+ }}
+pcx::check 0.8 std ::csv::join \
+ {checkSimpleArgs 1 4 {
+ checkList
+ checkWord
+ checkWord
+ {checkKeyword 1 {auto always}}
+ }}
+pcx::check 0.7 std ::csv::joinlist \
+ {checkSimpleArgs 1 3 {
+ checkList
+ checkWord
+ checkWord
+ }}
+pcx::check 0.8 std ::csv::joinlist \
+ {checkSimpleArgs 1 4 {
+ checkList
+ checkWord
+ checkWord
+ {checkKeyword 1 {auto always}}
+ }}
+pcx::check 0.7 std ::csv::joinmatrix \
+ {checkSimpleArgs 1 3 {
+ checkWord
+ checkWord
+ checkWord
+ }}
+pcx::check 0.8 std ::csv::joinmatrix \
+ {checkSimpleArgs 1 4 {
+ checkWord
+ checkWord
+ checkWord
+ {checkKeyword 1 {auto always}}
+ }}
+pcx::check 0.7 std ::csv::read2matrix \
+ {checkSimpleArgs 2 -1 {
+ {checkSwitches 1 {
+ -alternate
+ } {checkSimpleArgs 1 4 {
+ checkChannelID
+ checkWord
+ checkWord
+ {checkKeyword 1 {none empty auto}}
+ }}}
+ }}
+pcx::check 0.7 std ::csv::read2queue \
+ {checkSimpleArgs 2 -1 {
+ {checkSwitches 1 {
+ -alternate
+ } {checkSimpleArgs 2 3 {
+ checkChannelID
+ checkWord
+ checkWord
+ }}}
+ }}
+pcx::check 0.7 std ::csv::report \
+ {checkSimpleArgs 2 3 {
+ {checkOption {
+ {printmatrix {checkSimpleArgs 1 1 {
+ checkWord
+ }}}
+ {printmatrix2channel {checkSimpleArgs 2 2 {
+ checkWord
+ checkChannelID
+ }}}
+ } {}}
+ }}
+pcx::check 0.7 std ::csv::split \
+ {checkSimpleArgs 1 -1 {
+ {checkSwitches 1 {
+ -alternate
+ } {checkSimpleArgs 1 3 {
+ checkWord
+ checkWord
+ checkWord
+ }}}
+ }}
+pcx::check 0.7 std ::csv::split2matrix \
+ {checkSimpleArgs 2 -1 {
+ {checkSwitches 1 {
+ -alternate
+ } {checkSimpleArgs 1 4 {
+ checkChannelID
+ checkWord
+ checkWord
+ {checkKeyword 1 {none empty auto}}
+ }}}
+ }}
+pcx::check 0.7 std ::csv::split2queue \
+ {checkSimpleArgs 2 -1 {
+ {checkSwitches 1 {
+ -alternate
+ } {checkSimpleArgs 2 3 {
+ checkChannelID
+ checkWord
+ checkWord
+ }}}
+ }}
+pcx::check 0.7 std ::csv::writematrix \
+ {checkSimpleArgs 2 4 {
+ checkWord
+ checkChannelID
+ checkWord
+ checkWord
+ }}
+pcx::check 0.7 std ::csv::writequeue \
+ {checkSimpleArgs 2 4 {
+ checkWord
+ checkChannelID
+ checkWord
+ checkWord
+ }}
+
+# Initialization via pcx::init.
+# Use a ::csv::init procedure for non-standard initialization.
+pcx::complete
diff --git a/tcllib/modules/csv/csv.tcl b/tcllib/modules/csv/csv.tcl
new file mode 100644
index 0000000..a76336f
--- /dev/null
+++ b/tcllib/modules/csv/csv.tcl
@@ -0,0 +1,789 @@
+# csv.tcl --
+#
+# Tcl implementations of CSV reader and writer
+#
+# Copyright (c) 2001 by Jeffrey Hobbs
+# Copyright (c) 2001-2013 by Andreas Kupries <andreas_kupries@users.sourceforge.net>
+#
+# See the file "license.terms" for information on usage and redistribution
+# of this file, and for a DISCLAIMER OF ALL WARRANTIES.
+#
+# RCS: @(#) $Id: csv.tcl,v 1.28 2011/11/23 02:22:10 andreas_kupries Exp $
+
+package require Tcl 8.4
+package provide csv 0.8.1
+
+namespace eval ::csv {
+ namespace export join joinlist read2matrix read2queue report
+ namespace export split split2matrix split2queue writematrix writequeue
+}
+
+# ::csv::join --
+#
+# Takes a list of values and generates a string in CSV format.
+#
+# Arguments:
+# values A list of the values to join
+# sepChar The separator character, defaults to comma
+# delChar The delimiter character, defaults to quote
+# delMode If set to 'always', values are always surrounded by delChar
+#
+# Results:
+# A string containing the values in CSV format.
+
+proc ::csv::join {values {sepChar ,} {delChar \"} {delMode auto}} {
+ set out ""
+ set sep {}
+ foreach val $values {
+ if {($delMode eq "always") || [string match "*\[${delChar}$sepChar\r\n\]*" $val]} {
+ append out $sep${delChar}[string map [list $delChar ${delChar}${delChar}] $val]${delChar}
+ } else {
+ append out $sep${val}
+ }
+ set sep $sepChar
+ }
+ return $out
+}
+
+# ::csv::joinlist --
+#
+# Takes a list of lists of values and generates a string in CSV
+# format. Each item in the list is made into a single CSV
+# formatted record in the final string, the records being
+# separated by newlines.
+#
+# Arguments:
+# values A list of the lists of the values to join
+# sepChar The separator character, defaults to comma
+# delChar The delimiter character, defaults to quote
+# delMode If set to 'always', values are always surrounded by delChar
+#
+# Results:
+# A string containing the values in CSV format, the records
+# separated by newlines.
+
+proc ::csv::joinlist {values {sepChar ,} {delChar \"} {delMode auto}} {
+ set out ""
+ foreach record $values {
+ # note that this is ::csv::join
+ append out "[join $record $sepChar $delChar $delMode]\n"
+ }
+ return $out
+}
+
+# ::csv::joinmatrix --
+#
+# Takes a matrix object following the API specified for the
+# struct::matrix package. Each row of the matrix is converted
+# into a single CSV formatted record in the final string, the
+# records being separated by newlines.
+#
+# Arguments:
+# matrix Matrix object command.
+# sepChar The separator character, defaults to comma
+# delChar The delimiter character, defaults to quote
+# delMode If set to 'always', values are always surrounded by delChar
+#
+# Results:
+# A string containing the values in CSV format, the records
+# separated by newlines.
+
+proc ::csv::joinmatrix {matrix {sepChar ,} {delChar \"} {delMode auto}} {
+ return [joinlist [$matrix get rect 0 0 end end] $sepChar $delChar $delMode]
+}
+
+# ::csv::iscomplete --
+#
+# A predicate checking if the argument is a complete csv record.
+#
+# Arguments
+# data The (partial) csv record to check.
+#
+# Results:
+# A boolean flag indicating the completeness of the data. The
+# result is true if the data is complete.
+
+proc ::csv::iscomplete {data} {
+ expr {1 - [regexp -all \" $data] % 2}
+}
+
+# ::csv::read2matrix --
+#
+# A wrapper around "Split2matrix" reading CSV formatted
+# lines from the specified channel and adding it to the given
+# matrix.
+#
+# Arguments:
+# m The matrix to add the read data too.
+# chan The channel to read from.
+# sepChar The separator character, defaults to comma
+# expand The expansion mode. The default is none
+#
+# Results:
+# A list of the values in 'line'.
+
+proc ::csv::read2matrix {args} {
+ # FR #481023
+ # See 'split2matrix' for the available expansion modes.
+
+ # Argument syntax:
+ #
+ #2) chan m
+ #3) chan m sepChar
+ #3) -alternate chan m
+ #4) -alternate chan m sepChar
+ #4) chan m sepChar expand
+ #5) -alternate chan m sepChar expand
+
+ set alternate 0
+ set sepChar ,
+ set expand none
+
+ switch -exact -- [llength $args] {
+ 2 {
+ foreach {chan m} $args break
+ }
+ 3 {
+ foreach {a b c} $args break
+ if {[string equal $a "-alternate"]} {
+ set alternate 1
+ set chan $b
+ set m $c
+ } else {
+ set chan $a
+ set m $b
+ set sepChar $c
+ }
+ }
+ 4 {
+ foreach {a b c d} $args break
+ if {[string equal $a "-alternate"]} {
+ set alternate 1
+ set chan $b
+ set m $c
+ set sepChar $d
+ } else {
+ set chan $a
+ set m $b
+ set sepChar $c
+ set expand $d
+ }
+ }
+ 5 {
+ foreach {a b c d e} $args break
+ if {![string equal $a "-alternate"]} {
+ return -code error "wrong#args: Should be ?-alternate? chan m ?separator? ?expand?"
+ }
+ set alternate 1
+
+ set chan $b
+ set m $c
+ set sepChar $d
+ set expand $e
+ }
+ 0 - 1 -
+ default {
+ return -code error "wrong#args: Should be ?-alternate? chan m ?separator? ?expand?"
+ }
+ }
+
+ if {[string length $sepChar] < 1} {
+ return -code error "illegal separator character \"$sepChar\", is empty"
+ } elseif {[string length $sepChar] > 1} {
+ return -code error "illegal separator character \"$sepChar\", is a string"
+ }
+
+ set data ""
+ while {![eof $chan]} {
+ if {[gets $chan line] < 0} {continue}
+
+ # Why skip empty lines? They may be in data. Except if the
+ # buffer is empty, i.e. we are between records.
+ if {$line == {} && $data == {}} {continue}
+
+ append data $line
+ if {![iscomplete $data]} {
+ # Odd number of quotes - must have embedded newline
+ append data \n
+ continue
+ }
+
+ Split2matrix $alternate $m $data $sepChar $expand
+ set data ""
+ }
+ return
+}
+
+# ::csv::read2queue --
+#
+# A wrapper around "::csv::split2queue" reading CSV formatted
+# lines from the specified channel and adding it to the given
+# queue.
+#
+# Arguments:
+# q The queue to add the read data too.
+# chan The channel to read from.
+# sepChar The separator character, defaults to comma
+#
+# Results:
+# A list of the values in 'line'.
+
+proc ::csv::read2queue {args} {
+ # Argument syntax:
+ #
+ #2) chan q
+ #3) chan q sepChar
+ #3) -alternate chan q
+ #4) -alternate chan q sepChar
+
+ set alternate 0
+ set sepChar ,
+
+ switch -exact -- [llength $args] {
+ 2 {
+ foreach {chan q} $args break
+ }
+ 3 {
+ foreach {a b c} $args break
+ if {[string equal $a "-alternate"]} {
+ set alternate 1
+ set chan $b
+ set q $c
+ } else {
+ set chan $a
+ set q $b
+ set sepChar $c
+ }
+ }
+ 4 {
+ foreach {a b c d} $args break
+ if {![string equal $a "-alternate"]} {
+ return -code error "wrong#args: Should be ?-alternate? chan q ?separator?"
+ }
+ set alternate 1
+ set chan $b
+ set q $c
+ set sepChar $d
+ }
+ 0 - 1 -
+ default {
+ return -code error "wrong#args: Should be ?-alternate? chan q ?separator?"
+ }
+ }
+
+ if {[string length $sepChar] < 1} {
+ return -code error "illegal separator character \"$sepChar\", is empty"
+ } elseif {[string length $sepChar] > 1} {
+ return -code error "illegal separator character \"$sepChar\", is a string"
+ }
+
+ set data ""
+ while {![eof $chan]} {
+ if {[gets $chan line] < 0} {continue}
+
+ # Why skip empty lines? They may be in data. Except if the
+ # buffer is empty, i.e. we are between records.
+ if {$line == {} && $data == {}} {continue}
+
+ append data $line
+ if {![iscomplete $data]} {
+ # Odd number of quotes - must have embedded newline
+ append data \n
+ continue
+ }
+
+ $q put [Split $alternate $data $sepChar]
+ set data ""
+ }
+ return
+}
+
+# ::csv::report --
+#
+# A report command which can be used by the matrix methods
+# "format-via" and "format2chan-via". For the latter this
+# command delegates the work to "::csv::writematrix". "cmd" is
+# expected to be either "printmatrix" or
+# "printmatrix2channel". The channel argument, "chan", has to
+# be present for the latter and must not be present for the first.
+#
+# Arguments:
+# cmd Either 'printmatrix' or 'printmatrix2channel'
+# matrix The matrix to format.
+# args 0 (chan): The channel to write to
+#
+# Results:
+# None for 'printmatrix2channel', else the CSV formatted string.
+
+proc ::csv::report {cmd matrix args} {
+ switch -exact -- $cmd {
+ printmatrix {
+ if {[llength $args] > 0} {
+ return -code error "wrong # args:\
+ ::csv::report printmatrix matrix"
+ }
+ return [joinlist [$matrix get rect 0 0 end end]]
+ }
+ printmatrix2channel {
+ if {[llength $args] != 1} {
+ return -code error "wrong # args:\
+ ::csv::report printmatrix2channel matrix chan"
+ }
+ writematrix $matrix [lindex $args 0]
+ return ""
+ }
+ default {
+ return -code error "Unknown method $cmd"
+ }
+ }
+}
+
+# ::csv::split --
+#
+# Split a string according to the rules for CSV processing.
+# This assumes that the string contains a single line of CSVs
+#
+# Arguments:
+# line The string to split
+# sepChar The separator character, defaults to comma
+#
+# Results:
+# A list of the values in 'line'.
+
+proc ::csv::split {args} {
+ # Argument syntax:
+ #
+ # (1) line
+ # (2) line sepChar
+ # (2) -alternate line
+ # (3) -alternate line sepChar
+
+ # (3) line sepChar delChar
+ # (4) -alternate line sepChar delChar
+
+ set alternate 0
+ set sepChar ,
+ set delChar \"
+
+ switch -exact -- [llength $args] {
+ 1 {
+ set line [lindex $args 0]
+ }
+ 2 {
+ foreach {a b} $args break
+ if {[string equal $a "-alternate"]} {
+ set alternate 1
+ set line $b
+ } else {
+ set line $a
+ set sepChar $b
+ }
+ }
+ 3 {
+ foreach {a b c} $args break
+ if {[string equal $a "-alternate"]} {
+ set alternate 1
+ set line $b
+ set sepChar $c
+ } else {
+ set line $a
+ set sepChar $b
+ set delChar $c
+ }
+ }
+ 4 {
+ foreach {a b c d} $args break
+ if {![string equal $a "-alternate"]} {
+ return -code error "wrong#args: Should be ?-alternate? line ?separator? ?delimiter?"
+ }
+ set alternate 1
+ set line $b
+ set sepChar $c
+ set delChar $d
+ }
+ 0 -
+ default {
+ return -code error "wrong#args: Should be ?-alternate? line ?separator? ?delimiter?"
+ }
+ }
+
+ if {[string length $sepChar] < 1} {
+ return -code error "illegal separator character ${delChar}$sepChar${delChar}, is empty"
+ } elseif {[string length $sepChar] > 1} {
+ return -code error "illegal separator character ${delChar}$sepChar${delChar}, is a string"
+ }
+
+ if {[string length $delChar] < 1} {
+ return -code error "illegal separator character \"$delChar\", is empty"
+ } elseif {[string length $delChar] > 1} {
+ return -code error "illegal separator character \"$delChar\", is a string"
+ }
+
+ return [Split $alternate $line $sepChar $delChar]
+}
+
+proc ::csv::Split {alternate line sepChar {delChar \"}} {
+ # Protect the sepchar from special interpretation by
+ # the regex calls below.
+
+ set sepRE \[\[.${sepChar}.]]
+ set delRE \[\[.${delChar}.]]
+
+ if {$alternate} {
+ # The alternate syntax requires a different parser.
+ # A variation of the string map / regsub parser for the
+ # regular syntax was tried but does not handle embedded
+ # doubled " well (testcase csv-91.3 was 'knownBug', sole
+ # one, still a bug). Now we just tokenize the input into
+ # the primary parts (sep char, "'s and the rest) and then
+ # use an explicitly coded state machine (DFA) to parse
+ # and convert token sequences.
+
+ ## puts 1->>$line<<
+ set line [string map [list \
+ $sepChar \0$sepChar\0 \
+ $delChar \0${delChar}\0 \
+ ] $line]
+
+ ## puts 2->>$line<<
+ set line [string map [list \0\0 \0] $line]
+ regsub "^\0" $line {} line
+ regsub "\0$" $line {} line
+
+ ## puts 3->>$line<<
+
+ set val ""
+ set res ""
+ set state base
+
+ ## puts 4->>[::split $line \0]
+ foreach token [::split $line \0] {
+
+ ## puts "\t*= $state\t>>$token<<"
+ switch -exact -- $state {
+ base {
+ if {[string equal $token "${delChar}"]} {
+ set state qvalue
+ continue
+ }
+ if {[string equal $token $sepChar]} {
+ lappend res $val
+ set val ""
+ continue
+ }
+ append val $token
+ }
+ qvalue {
+ if {[string equal $token "${delChar}"]} {
+ # May end value, may be a doubled "
+ set state endordouble
+ continue
+ }
+ append val $token
+ }
+ endordouble {
+ if {[string equal $token "${delChar}"]} {
+ # Doubled ", append to current value
+ append val ${delChar}
+ set state qvalue
+ continue
+ }
+ # Last " was end of quoted value. Close it.
+ # We expect current as $sepChar
+
+ lappend res $val
+ set val ""
+ set state base
+
+ if {[string equal $token $sepChar]} {continue}
+
+ # Undoubled " in middle of text. Just assume that
+ # remainder is another qvalue.
+ set state qvalue
+ }
+ default {
+ return -code error "Internal error, illegal parsing state"
+ }
+ }
+ }
+
+ ## puts "/= $state\t>>$val<<"
+
+ lappend res $val
+
+ ## puts 5->>$res<<
+ return $res
+ } else {
+ regsub -- "$sepRE${delRE}${delRE}$" $line $sepChar\0${delChar}${delChar}\0 line
+ regsub -- "^${delRE}${delRE}$sepRE" $line \0${delChar}${delChar}\0$sepChar line
+ regsub -all -- {(^${delChar}|${delChar}$)} $line \0 line
+
+ set line [string map [list \
+ $sepChar${delChar}${delChar}${delChar} $sepChar\0${delChar} \
+ ${delChar}${delChar}${delChar}$sepChar ${delChar}\0$sepChar \
+ ${delChar}${delChar} ${delChar} \
+ ${delChar} \0 \
+ ] $line]
+
+ set end 0
+ while {[regexp -indices -start $end -- {(\0)[^\0]*(\0)} $line \
+ -> start end]} {
+ set start [lindex $start 0]
+ set end [lindex $end 0]
+ set range [string range $line $start $end]
+ if {[string first $sepChar $range] >= 0} {
+ set line [string replace $line $start $end \
+ [string map [list $sepChar \1] $range]]
+ }
+ incr end
+ }
+ set line [string map [list $sepChar \0 \1 $sepChar \0 {} ] $line]
+ return [::split $line \0]
+
+ }
+}
+
+# ::csv::split2matrix --
+#
+# Split a string according to the rules for CSV processing.
+# This assumes that the string contains a single line of CSVs.
+# The resulting list of values is appended to the specified
+# matrix, as a new row. The code assumes that the matrix provides
+# the same interface as the queue provided by the 'struct'
+# module of tcllib, "add row" in particular.
+#
+# Arguments:
+# m The matrix to write the resulting list to.
+# line The string to split
+# sepChar The separator character, defaults to comma
+# expand The expansion mode. The default is none
+#
+# Results:
+# A list of the values in 'line', written to 'q'.
+
+proc ::csv::split2matrix {args} {
+ # FR #481023
+
+ # Argument syntax:
+ #
+ #2) m line
+ #3) m line sepChar
+ #3) -alternate m line
+ #4) -alternate m line sepChar
+ #4) m line sepChar expand
+ #5) -alternate m line sepChar expand
+
+ set alternate 0
+ set sepChar ,
+ set expand none
+
+ switch -exact -- [llength $args] {
+ 2 {
+ foreach {m line} $args break
+ }
+ 3 {
+ foreach {a b c} $args break
+ if {[string equal $a "-alternate"]} {
+ set alternate 1
+ set m $b
+ set line $c
+ } else {
+ set m $a
+ set line $b
+ set sepChar $c
+ }
+ }
+ 4 {
+ foreach {a b c d} $args break
+ if {[string equal $a "-alternate"]} {
+ set alternate 1
+ set m $b
+ set line $c
+ set sepChar $d
+ } else {
+ set m $a
+ set line $b
+ set sepChar $c
+ set expand $d
+ }
+ }
+ 4 {
+ foreach {a b c d e} $args break
+ if {![string equal $a "-alternate"]} {
+ return -code error "wrong#args: Should be ?-alternate? m line ?separator? ?expand?"
+ }
+ set alternate 1
+
+ set m $b
+ set line $c
+ set sepChar $d
+ set expand $e
+ }
+ 0 - 1 -
+ default {
+ return -code error "wrong#args: Should be ?-alternate? m line ?separator? ?expand?"
+ }
+ }
+
+ if {[string length $sepChar] < 1} {
+ return -code error "illegal separator character \"$sepChar\", is empty"
+ } elseif {[string length $sepChar] > 1} {
+ return -code error "illegal separator character \"$sepChar\", is a string"
+ }
+
+ Split2matrix $alternate $m $line $sepChar $expand
+ return
+}
+
+proc ::csv::Split2matrix {alternate m line sepChar expand} {
+ set csv [Split $alternate $line $sepChar]
+
+ # Expansion modes
+ # - none : default, behaviour of original implementation.
+ # no expansion is done, lines are silently truncated
+ # to the number of columns in the matrix.
+ #
+ # - empty : A matrix without columns is expanded to the number
+ # of columns in the first line added to it. All
+ # following lines are handled as if "mode == none"
+ # was set.
+ #
+ # - auto : Full auto-mode. The matrix is expanded as needed to
+ # hold all columns of all lines.
+
+ switch -exact -- $expand {
+ none {}
+ empty {
+ if {[$m columns] == 0} {
+ $m add columns [llength $csv]
+ }
+ }
+ auto {
+ if {[$m columns] < [llength $csv]} {
+ $m add columns [expr {[llength $csv] - [$m columns]}]
+ }
+ }
+ }
+ $m add row $csv
+ return
+}
+
+# ::csv::split2queue --
+#
+# Split a string according to the rules for CSV processing.
+# This assumes that the string contains a single line of CSVs.
+# The resulting list of values is appended to the specified
+# queue, as a single item. IOW each item in the queue represents
+# a single CSV record. The code assumes that the queue provides
+# the same interface as the queue provided by the 'struct'
+# module of tcllib, "put" in particular.
+#
+# Arguments:
+# q The queue to write the resulting list to.
+# line The string to split
+# sepChar The separator character, defaults to comma
+#
+# Results:
+# A list of the values in 'line', written to 'q'.
+
+proc ::csv::split2queue {args} {
+ # Argument syntax:
+ #
+ #2) q line
+ #3) q line sepChar
+ #3) -alternate q line
+ #4) -alternate q line sepChar
+
+ set alternate 0
+ set sepChar ,
+
+ switch -exact -- [llength $args] {
+ 2 {
+ foreach {q line} $args break
+ }
+ 3 {
+ foreach {a b c} $args break
+ if {[string equal $a "-alternate"]} {
+ set alternate 1
+ set q $b
+ set line $c
+ } else {
+ set q $a
+ set line $b
+ set sepChar $c
+ }
+ }
+ 4 {
+ foreach {a b c d} $args break
+ if {![string equal $a "-alternate"]} {
+ return -code error "wrong#args: Should be ?-alternate? q line ?separator?"
+ }
+ set alternate 1
+
+ set q $b
+ set line $c
+ set sepChar $d
+ }
+ 0 - 1 -
+ default {
+ return -code error "wrong#args: Should be ?-alternate? q line ?separator?"
+ }
+ }
+
+ if {[string length $sepChar] < 1} {
+ return -code error "illegal separator character \"$sepChar\", is empty"
+ } elseif {[string length $sepChar] > 1} {
+ return -code error "illegal separator character \"$sepChar\", is a string"
+ }
+
+ $q put [Split $alternate $line $sepChar]
+ return
+}
+
+# ::csv::writematrix --
+#
+# A wrapper around "::csv::join" taking the rows in a matrix and
+# writing them as CSV formatted lines into the channel.
+#
+# Arguments:
+# m The matrix to take the data to write from.
+# chan The channel to write into.
+# sepChar The separator character, defaults to comma
+#
+# Results:
+# None.
+
+proc ::csv::writematrix {m chan {sepChar ,} {delChar \"}} {
+ set n [$m rows]
+ for {set r 0} {$r < $n} {incr r} {
+ puts $chan [join [$m get row $r] $sepChar $delChar]
+ }
+
+ # Memory intensive alternative:
+ # puts $chan [joinlist [m get rect 0 0 end end] $sepChar $delChar]
+ return
+}
+
+# ::csv::writequeue --
+#
+# A wrapper around "::csv::join" taking the rows in a queue and
+# writing them as CSV formatted lines into the channel.
+#
+# Arguments:
+# q The queue to take the data to write from.
+# chan The channel to write into.
+# sepChar The separator character, defaults to comma
+#
+# Results:
+# None.
+
+proc ::csv::writequeue {q chan {sepChar ,} {delChar \"}} {
+ while {[$q size] > 0} {
+ puts $chan [join [$q get] $sepChar $delChar]
+ }
+
+ # Memory intensive alternative:
+ # puts $chan [joinlist [$q get [$q size]] $sepChar $delChar]
+ return
+}
+
diff --git a/tcllib/modules/csv/csv.test b/tcllib/modules/csv/csv.test
new file mode 100644
index 0000000..1e2ff78
--- /dev/null
+++ b/tcllib/modules/csv/csv.test
@@ -0,0 +1,998 @@
+# -*- tcl -*-
+# Tests for the find function.
+#
+# Sourcing this file into Tcl runs the tests and generates output for errors.
+# No output means no errors were found.
+#
+# Copyright (c) 2001-2011 by Andreas Kupries <andreas_kupries@users.sourceforge.net>
+# All rights reserved.
+#
+# RCS: @(#) $Id: csv.test,v 1.23 2011/11/23 02:22:10 andreas_kupries Exp $
+
+# -------------------------------------------------------------------------
+
+source [file join \
+ [file dirname [file dirname [file join [pwd] [info script]]]] \
+ devtools testutilities.tcl]
+
+testsNeedTcl 8.3
+testsNeedTcltest 1.0
+
+support {
+ use struct/queue.tcl struct::queue
+ use struct/matrix.tcl struct::matrix
+}
+testing {
+ useLocal csv.tcl csv
+}
+
+# -------------------------------------------------------------------------
+
+set str1 {"123","""a""",,hello}
+set str2 {1," o, ""a"" ,b ", 3}
+set str3 {"1"," o, "","" ,b ", 3}
+set str4 {1," foo,bar,baz", 3}
+set str5 {1,"""""a""""",b}
+set str6 {123,"123,521.2","Mary says ""Hello, I am Mary"""}
+
+set str1a {123,"""a""",,hello}
+set str3a {1," o, "","" ,b ", 3}
+
+# Custom delimiter, =
+
+set str1_ {=123=,===a===,,hello}
+set str2_ {1,= o, ==a== ,b =, 3}
+set str3_ {=1=,= o, ==,== ,b =, 3}
+set str4_ {1,= foo,bar,baz=, 3}
+set str5_ {1,=====a=====,b}
+set str6_ {123,=123,521.2=,=Mary says "Hello, I am Mary"=}
+
+set str1a_ {123,===a===,,hello}
+set str3a_ {1,= o, ==,== ,b =, 3}
+
+set str7 {=1=,=====a=====,=b=}
+
+# -------------------------------------------------------------------------
+
+test csv-1.1 {split} {
+ csv::split $str1
+} {123 {"a"} {} hello}
+
+test csv-1.2 {split} {
+ csv::split $str2
+} {1 { o, "a" ,b } { 3}}
+
+test csv-1.3 {split} {
+ csv::split $str3
+} {1 { o, "," ,b } { 3}}
+
+test csv-1.4 {split} {
+ csv::split $str4
+} {1 { foo,bar,baz} { 3}}
+
+test csv-1.5 {split} {
+ csv::split $str5
+} {1 {""a""} b}
+
+test csv-1.6 {split} {
+ csv::split $str6
+} {123 123,521.2 {Mary says "Hello, I am Mary"}}
+
+test csv-1.7 {split on join} {
+ # csv 0.1 was exposed to the RE \A matching problem with regsub -all
+ set x [list "\"hello, you\"" a b c]
+ ::csv::split [::csv::join $x]
+} [list "\"hello, you\"" a b c]
+
+test csv-1.8-1 {split empty fields} {
+ csv::split {1 2 "" ""} { }
+} {1 2 {"} {"}}
+
+test csv-1.9-1 {split empty fields} {
+ csv::split {1 2 3 ""} { }
+} {1 2 3 {"}}
+
+test csv-1.10-1 {split empty fields} {
+ csv::split {"" "" 1 2} { }
+} {{"} {"} 1 2}
+
+test csv-1.11-1 {split empty fields} {
+ csv::split {"" 0 1 2} { }
+} {{"} 0 1 2}
+
+test csv-1.12-1 {split empty fields} {
+ csv::split {"" ""} { }
+} {{"} {"}}
+
+test csv-1.13-1 {split empty fields} {
+ csv::split {"" "" ""} { }
+} {{"} {"} {"}}
+
+test csv-1.14-1 {split empty fields} {
+ csv::split {"" 0 "" 2} { }
+} {{"} 0 {"} 2}
+
+test csv-1.15-1 {split empty fields} {
+ csv::split {1 "" 3 ""} { }
+} {1 {"} 3 {"}}
+
+test csv-1.8-2 {split empty fields} {
+ csv::split "1,2,,"
+} {1 2 {} {}}
+
+test csv-1.9-2 {split empty fields} {
+ csv::split "1,2,3,"
+} {1 2 3 {}}
+
+test csv-1.10-2 {split empty fields} {
+ csv::split ",,1,2"
+} {{} {} 1 2}
+
+test csv-1.11-2 {split empty fields} {
+ csv::split ",0,1,2"
+} {{} 0 1 2}
+
+test csv-1.12-2 {split empty fields} {
+ csv::split ","
+} {{} {}}
+
+test csv-1.13-2 {split empty fields} {
+ csv::split ",,"
+} {{} {} {}}
+
+test csv-1.14-2 {split empty fields} {
+ csv::split ",0,,2"
+} {{} 0 {} 2}
+
+test csv-1.15-2 {split empty fields} {
+ csv::split "1,,3,"
+} {1 {} 3 {}}
+
+test csv-1.8-3 {split empty fields} {
+ csv::split {1 2 } { }
+} {1 2 {} {}}
+
+test csv-1.9-3 {split empty fields} {
+ csv::split {1 2 3 } { }
+} {1 2 3 {}}
+
+test csv-1.10-3 {split empty fields} {
+ csv::split { 1 2} { }
+} {{} {} 1 2}
+
+test csv-1.11-3 {split empty fields} {
+ csv::split { 0 1 2} { }
+} {{} 0 1 2}
+
+test csv-1.12-3 {split empty fields} {
+ csv::split { } { }
+} {{} {}}
+
+test csv-1.13-3 {split empty fields} {
+ csv::split { } { }
+} {{} {} {}}
+
+test csv-1.14-3 {split empty fields} {
+ csv::split { 0 2} { }
+} {{} 0 {} 2}
+
+test csv-1.15-3 {split empty fields} {
+ csv::split {1 3 } { }
+} {1 {} 3 {}}
+
+
+test csv-1.8-4 {split empty fields} {
+ csv::split {1,2,"",""}
+} {1 2 {"} {"}}
+
+test csv-1.9-4 {split empty fields} {
+ csv::split {1,2,3,""}
+} {1 2 3 {"}}
+
+test csv-1.10-4 {split empty fields} {
+ csv::split {"","",1,2}
+} {{"} {"} 1 2}
+
+test csv-1.11-4 {split empty fields} {
+ csv::split {"",0,1,2}
+} {{"} 0 1 2}
+
+test csv-1.12-4 {split empty fields} {
+ csv::split {"",""}
+} {{"} {"}}
+
+test csv-1.13-4 {split empty fields} {
+ csv::split {"","",""}
+} {{"} {"} {"}}
+
+test csv-1.14-4 {split empty fields} {
+ csv::split {"",0,"",2}
+} {{"} 0 {"} 2}
+
+test csv-1.15-4 {split empty fields} {
+ csv::split {1,"",3,""}
+} {1 {"} 3 {"}}
+
+# Try various separator characters
+
+foreach {n sep} {
+ 0 | 1 + 2 *
+ 3 / 4 \ 5 [
+ 6 ] 7 ( 8 )
+ 9 ? 10 , 11 ;
+ 12 . 13 - 14 =
+ 15 : 16 x 17 9
+} {
+ test csv-1.16-$n "split on $sep" {
+ ::csv::split [join [list REC DPI AD1 AD2 AD3] $sep] $sep
+ } {REC DPI AD1 AD2 AD3}
+}
+
+test csv-2.1 {join} {
+ csv::join {123 {"a"} {} hello}
+} $str1a
+
+test csv-2.2 {join} {
+ csv::join {1 { o, "a" ,b } { 3}}
+} $str2
+
+test csv-2.3 {join} {
+ csv::join {1 { o, "," ,b } { 3}}
+} $str3a
+
+test csv-2.4 {join} {
+ csv::join {1 { foo,bar,baz} { 3}}
+} $str4
+
+test csv-2.5 {join} {
+ csv::join {1 {""a""} b}
+} $str5
+
+test csv-2.6 {join} {
+ csv::join {123 123,521.2 {Mary says "Hello, I am Mary"}}
+} $str6
+
+test csv-2.7 {join, custom delimiter} {
+ csv::join {123 =a= {} hello} , =
+} $str1a_
+
+test csv-2.8 {join, custom delimiter} {
+ csv::join {1 { o, =a= ,b } { 3}} , =
+} $str2_
+
+test csv-2.9 {join, custom delimiter} {
+ csv::join {1 { o, =,= ,b } { 3}} , =
+} $str3a_
+
+test csv-2.10 {join, custom delimiter} {
+ csv::join {1 { foo,bar,baz} { 3}} , =
+} $str4_
+
+test csv-2.11 {join, custom delimiter} {
+ csv::join {1 ==a== b} , =
+} $str5_
+
+test csv-2.12 {join, custom delimiter} {
+ csv::join {123 123,521.2 {Mary says "Hello, I am Mary"}} , =
+} $str6_
+
+test csv-2.13-sf-1724818 {join, newlines in string, sf bug 1724818} {
+ csv::join {123 {John Doe} "123 Main St.\nSmalltown, OH 44444"}
+} "123,John Doe,\"123 Main St.\nSmalltown, OH 44444\""
+
+test csv-2.14 {join, custom delimiter, always} {
+ csv::join {1 ==a== b} , = always
+} $str7
+
+# Malformed inputs
+
+test csv-3.1 {split} {
+ csv::split {abcd,abc",abc} ; # "
+} {abcd abc abc}
+
+test csv-3.2 {split} {
+ csv::split {abcd,abc"",abc}
+} {abcd abc\" abc}
+
+
+test csv-4.1 {joinlist} {
+ csv::joinlist [list \
+ {123 {"a"} {} hello} \
+ {1 { o, "a" ,b } { 3}} \
+ {1 { o, "," ,b } { 3}} \
+ {1 { foo,bar,baz} { 3}} \
+ {1 {""a""} b} \
+ {123 123,521.2 {Mary says "Hello, I am Mary"}}]
+} "$str1a\n$str2\n$str3a\n$str4\n$str5\n$str6\n"
+
+test csv-4.2 {joinlist, sepChar} {
+ csv::joinlist [list [list a b c] [list d e f]] @
+} "a@b@c\nd@e@f\n"
+
+test csv-4.3 {joinlist, custom delimiter} {
+ csv::joinlist [list \
+ {123 =a= {} hello} \
+ {1 { o, =a= ,b } { 3}} \
+ {1 { o, =,= ,b } { 3}} \
+ {1 { foo,bar,baz} { 3}} \
+ {1 ==a== b} \
+ {123 123,521.2 {Mary says "Hello, I am Mary"}}] , =
+} "$str1a_\n$str2_\n$str3a_\n$str4_\n$str5_\n$str6_\n"
+
+test csv-4.4 {joinlist, sepChar, custom delimiter} {
+ csv::joinlist [list [list a b c] [list d e f]] @ =
+} "a@b@c\nd@e@f\n"
+
+
+test csv-5.0.0 {reading csv files, bad separator, empty} {
+ ::struct::queue q
+ catch {::csv::read2queue dummy q {}} result
+ q destroy
+ set result
+} {illegal separator character "", is empty}
+
+test csv-5.0.1 {reading csv files, bad separator, string} {
+ ::struct::queue q
+ catch {::csv::read2queue dummy q foo} result
+ q destroy
+ set result
+} {illegal separator character "foo", is a string}
+
+test csv-5.0.2 {reading csv files, bad separator, empty} {
+ ::struct::matrix m
+ catch {::csv::read2matrix dummy m {}} result
+ m destroy
+ set result
+} {illegal separator character "", is empty}
+
+test csv-5.0.3 {reading csv files, bad separator, string} {
+ ::struct::matrix m
+ catch {::csv::read2matrix dummy m foo} result
+ m destroy
+ set result
+} {illegal separator character "foo", is a string}
+
+test csv-5.1 {reading csv files} {
+ set f [open [file join $::tcltest::testsDirectory mem_debug_bench.csv] r]
+ ::struct::queue q
+ ::csv::read2queue $f q
+ close $f
+ set result [list [q size] [q get 2]]
+ q destroy
+ set result
+} {251 {{000 VERSIONS: 2:8.4a3 1:8.4a3 1:8.4a3%} {001 {CATCH return ok} 7 13 53.85}}}
+
+test csv-5.2 {reading csv files} {
+ set f [open [file join $::tcltest::testsDirectory mem_debug_bench_a.csv] r]
+ ::struct::queue q
+ ::csv::read2queue $f q
+ close $f
+ set result [list [q size] [q get 2]]
+ q destroy
+ set result
+} {251 {{000 VERSIONS: 2:8.4a3 1:8.4a3 1:8.4a3%} {001 {CATCH return ok} 7 13 53.85}}}
+
+test csv-5.3 {reading csv files} {
+ set f [open [file join $::tcltest::testsDirectory mem_debug_bench.csv] r]
+ ::struct::matrix m
+ m add columns 5
+ ::csv::read2matrix $f m
+ close $f
+ set result [m get rect 0 227 end 231]
+ m destroy
+ set result
+} {{227 {STR append (1MB + 1MB * 3)} 125505 327765 38.29} {228 {STR append (1MB + 1MB * 5)} 158507 855295 18.53} {229 {STR append (1MB + (1b + 1K + 1b) * 100)} 33101 174031 19.02} {230 {STR info locals match} 946 1521 62.20} {231 {TRACE no trace set} 34 121 28.10}}
+
+test csv-5.4 {reading csv files} {
+ set f [open [file join $::tcltest::testsDirectory mem_debug_bench_a.csv] r]
+ ::struct::matrix m
+ m add columns 5
+ ::csv::read2matrix $f m
+ close $f
+ set result [m get rect 0 227 end 231]
+ m destroy
+ set result
+} {{227 {STR append (1MB + 1MB * 3)} 125505 327765 38.29} {228 {STR append (1MB + 1MB * 5)} 158507 855295 18.53} {229 {STR append (1MB + (1b + 1K + 1b) * 100)} 33101 174031 19.02} {230 {STR info locals match} 946 1521 62.20} {231 {TRACE no trace set} 34 121 28.10}}
+
+test csv-5.5 {reading csv files} {
+ set f [open [file join $::tcltest::testsDirectory mem_debug_bench.csv] r]
+ ::struct::matrix m
+ m add columns 5
+ ::csv::read2matrix $f m
+ close $f
+
+ set result [list]
+ foreach c {0 1 2 3 4} {
+ lappend result [m columnwidth $c]
+ }
+ m destroy
+ set result
+} {3 39 7 7 8}
+
+test csv-5.6 {reading csv files, linking} {
+ set f [open [file join $::tcltest::testsDirectory mem_debug_bench.csv] r]
+ ::struct::matrix m
+ m add columns 5
+ ::csv::read2matrix $f m
+ close $f
+ m link a
+ set result [array size a]
+ m destroy
+ set result
+} {1255}
+
+
+test csv-5.7 {reading csv files, empty expansion mode} {
+ set f [open [file join $::tcltest::testsDirectory mem_debug_bench.csv] r]
+ ::struct::matrix m
+ ::csv::read2matrix $f m , empty
+ close $f
+ set result [m get rect 0 227 end 231]
+ m destroy
+ set result
+} {{227 {STR append (1MB + 1MB * 3)} 125505 327765 38.29} {228 {STR append (1MB + 1MB * 5)} 158507 855295 18.53} {229 {STR append (1MB + (1b + 1K + 1b) * 100)} 33101 174031 19.02} {230 {STR info locals match} 946 1521 62.20} {231 {TRACE no trace set} 34 121 28.10}}
+
+test csv-5.8 {reading csv files, auto expansion mode} {
+ set f [open [file join $::tcltest::testsDirectory mem_debug_bench.csv] r]
+ ::struct::matrix m
+ m add columns 1
+ ::csv::read2matrix $f m , auto
+ close $f
+ set result [m get rect 0 227 end 231]
+ m destroy
+ set result
+} {{227 {STR append (1MB + 1MB * 3)} 125505 327765 38.29} {228 {STR append (1MB + 1MB * 5)} 158507 855295 18.53} {229 {STR append (1MB + (1b + 1K + 1b) * 100)} 33101 174031 19.02} {230 {STR info locals match} 946 1521 62.20} {231 {TRACE no trace set} 34 121 28.10}}
+
+# =========================================================================
+# Bug 2926387
+
+test csv-5.9.0 {reading csv files, inner field newline processing, bug 2926387} {
+ set m [struct::matrix]
+ set f [open [file join $::tcltest::testsDirectory 2926387.csv] r]
+ csv::read2matrix $f $m , auto
+ close $f
+ set result [$m serialize]
+ $m destroy
+ set result
+} {2 3 {{a b c} {d {e,
+e} f}}}
+
+test csv-5.9.1 {reading csv files, inner field newline processing, bug 2926387} {
+ set q [struct::queue]
+ set f [open [file join $::tcltest::testsDirectory 2926387.csv] r]
+ csv::read2queue $f $q
+ close $f
+ set result [$q get [$q size]]
+ $q destroy
+ set result
+} {{a b c} {d {e,
+e} f}}
+
+# =========================================================================
+
+test csv-6.1 {writing csv files} {
+ set f [open [localPath eval.csv] r]
+ ::struct::matrix m
+ m add columns 5
+ ::csv::read2matrix $f m
+ close $f
+
+ set f [open [makeFile {} eval-out1.csv] w]
+ ::csv::writematrix m $f
+ close $f
+
+ set result [viewFile eval-out1.csv]
+ m destroy
+ removeFile eval-out1.csv
+ set result
+} {023,EVAL cmd eval in list obj var,26,45,57.78
+024,EVAL cmd eval as list,23,42,54.76
+025,EVAL cmd eval as string,53,92,57.61
+026,EVAL cmd and mixed lists,3805,11276,33.74
+027,EVAL list cmd and mixed lists,3812,11325,33.66
+028,EVAL list cmd and pure lists,592,1598,37.05}
+
+test csv-6.2 {writing csv files} {
+ set f [open [localPath eval.csv] r]
+ ::struct::queue q
+ ::csv::read2queue $f q
+ close $f
+
+ set f [open [makeFile {} eval-out2.csv] w]
+ ::csv::writequeue q $f
+ close $f
+
+ set result [viewFile eval-out2.csv]
+ q destroy
+ removeFile eval-out2.csv
+ set result
+} {023,EVAL cmd eval in list obj var,26,45,57.78
+024,EVAL cmd eval as list,23,42,54.76
+025,EVAL cmd eval as string,53,92,57.61
+026,EVAL cmd and mixed lists,3805,11276,33.74
+027,EVAL list cmd and mixed lists,3812,11325,33.66
+028,EVAL list cmd and pure lists,592,1598,37.05}
+
+
+test csv-7.1 {reporting} {
+ set f [open [localPath eval.csv] r]
+ ::struct::matrix m
+ m add columns 5
+ ::csv::read2matrix $f m
+ close $f
+
+ set result [m format 2string csv::report]
+ m destroy
+ set result
+} {023,EVAL cmd eval in list obj var,26,45,57.78
+024,EVAL cmd eval as list,23,42,54.76
+025,EVAL cmd eval as string,53,92,57.61
+026,EVAL cmd and mixed lists,3805,11276,33.74
+027,EVAL list cmd and mixed lists,3812,11325,33.66
+028,EVAL list cmd and pure lists,592,1598,37.05
+}
+
+test csv-7.2 {reporting} {
+ set f [open [localPath eval.csv] r]
+ ::struct::matrix m
+ m add columns 5
+ ::csv::read2matrix $f m
+ close $f
+
+ set f [open [makeFile {} eval-out3.csv] w]
+ m format 2chan csv::report $f
+ close $f
+
+ set result [viewFile eval-out3.csv]
+ m destroy
+ removeFile eval-out3.csv
+ set result
+} {023,EVAL cmd eval in list obj var,26,45,57.78
+024,EVAL cmd eval as list,23,42,54.76
+025,EVAL cmd eval as string,53,92,57.61
+026,EVAL cmd and mixed lists,3805,11276,33.74
+027,EVAL list cmd and mixed lists,3812,11325,33.66
+028,EVAL list cmd and pure lists,592,1598,37.05}
+
+
+test csv-7.3 {report error} {
+ catch {::csv::report printmatrix foomatrix blarg} msg
+ set msg
+} {wrong # args: ::csv::report printmatrix matrix}
+
+test csv-7.4 {report error} {
+ catch {::csv::report printmatrix2channel foomatrix} msg
+ set msg
+} {wrong # args: ::csv::report printmatrix2channel matrix chan}
+
+test csv-7.5 {report error} {
+ catch {::csv::report printmatrix2channel foomatrix foo bar} msg
+ set msg
+} {wrong # args: ::csv::report printmatrix2channel matrix chan}
+
+test csv-7.6 {report error} {
+ catch {::csv::report foocmd foomatrix} msg
+ set msg
+} {Unknown method foocmd}
+
+
+## ============================================================
+## Test new restrictions on argument syntax of split.
+
+test csv-8.0 {csv argument error} {
+ catch {::csv::split} msg
+ set msg
+} {wrong#args: Should be ?-alternate? line ?separator? ?delimiter?}
+
+test csv-8.1 {csv argument error} {
+ catch {::csv::split a b c d e} msg
+ set msg
+} {wrong#args: Should be ?-alternate? line ?separator? ?delimiter?}
+
+test csv-8.2 {csv argument error} {
+ catch {::csv::split -alternate b {}} msg
+ set msg
+} {illegal separator character "", is empty}
+
+test csv-8.3 {csv argument error} {
+ catch {::csv::split -alternate b foo} msg
+ set msg
+} {illegal separator character "foo", is a string}
+
+test csv-8.4 {csv argument error} {
+ catch {::csv::split b {}} msg
+ set msg
+} {illegal separator character "", is empty}
+
+test csv-8.5 {csv argument error} {
+ catch {::csv::split b foo} msg
+ set msg
+} {illegal separator character "foo", is a string}
+
+## ============================================================
+## Tests for alternate syntax.
+
+
+test csv-91.1 {split} {
+ csv::split -alternate $str1
+} {123 {"a"} {} hello}
+
+test csv-91.2 {split} {
+ csv::split -alternate $str2
+} {1 { o, "a" ,b } { 3}}
+
+test csv-91.3 {split} {
+ csv::split -alternate $str3
+} {1 { o, "," ,b } { 3}}
+
+test csv-91.4 {split} {
+ csv::split -alternate $str4
+} {1 { foo,bar,baz} { 3}}
+
+test csv-91.5 {split} {
+ csv::split -alternate $str5
+} {1 {""a""} b}
+
+test csv-91.6 {split} {
+ csv::split -alternate $str6
+} {123 123,521.2 {Mary says "Hello, I am Mary"}}
+
+test csv-91.7 {split on join} {
+ # csv 0.1 was exposed to the RE \A matching problem with regsub -all
+ set x [list "\"hello, you\"" a b c]
+ ::csv::split -alternate [::csv::join $x]
+} [list "\"hello, you\"" a b c]
+
+test csv-91.8-1 {split empty fields} {
+ csv::split -alternate {1 2 "" ""} { }
+} {1 2 {} {}}
+
+test csv-91.9-1 {split empty fields} {
+ csv::split -alternate {1 2 3 ""} { }
+} {1 2 3 {}}
+
+test csv-91.10-1 {split empty fields} {
+ csv::split -alternate {"" "" 1 2} { }
+} {{} {} 1 2}
+
+test csv-91.11-1 {split empty fields} {
+ csv::split -alternate {"" 0 1 2} { }
+} {{} 0 1 2}
+
+test csv-91.12-1 {split empty fields} {
+ csv::split -alternate {"" ""} { }
+} {{} {}}
+
+test csv-91.13-1 {split empty fields} {
+ csv::split -alternate {"" "" ""} { }
+} {{} {} {}}
+
+test csv-91.14-1 {split empty fields} {
+ csv::split -alternate {"" 0 "" 2} { }
+} {{} 0 {} 2}
+
+test csv-91.15-1 {split empty fields} {
+ csv::split -alternate {1 "" 3 ""} { }
+} {1 {} 3 {}}
+
+test csv-91.8-2 {split empty fields} {
+ csv::split -alternate "1,2,,"
+} {1 2 {} {}}
+
+test csv-91.9-2 {split empty fields} {
+ csv::split -alternate "1,2,3,"
+} {1 2 3 {}}
+
+test csv-91.10-2 {split empty fields} {
+ csv::split -alternate ",,1,2"
+} {{} {} 1 2}
+
+test csv-91.11-2 {split empty fields} {
+ csv::split -alternate ",0,1,2"
+} {{} 0 1 2}
+
+test csv-91.12-2 {split empty fields} {
+ csv::split -alternate ","
+} {{} {}}
+
+test csv-91.13-2 {split empty fields} {
+ csv::split -alternate ",,"
+} {{} {} {}}
+
+test csv-91.14-2 {split empty fields} {
+ csv::split -alternate ",0,,2"
+} {{} 0 {} 2}
+
+test csv-91.15-2 {split empty fields} {
+ csv::split -alternate "1,,3,"
+} {1 {} 3 {}}
+
+test csv-91.8-3 {split empty fields} {
+ csv::split -alternate {1 2 } { }
+} {1 2 {} {}}
+
+test csv-91.9-3 {split empty fields} {
+ csv::split -alternate {1 2 3 } { }
+} {1 2 3 {}}
+
+test csv-91.10-3 {split empty fields} {
+ csv::split -alternate { 1 2} { }
+} {{} {} 1 2}
+
+test csv-91.11-3 {split empty fields} {
+ csv::split -alternate { 0 1 2} { }
+} {{} 0 1 2}
+
+test csv-91.12-3 {split empty fields} {
+ csv::split -alternate { } { }
+} {{} {}}
+
+test csv-91.13-3 {split empty fields} {
+ csv::split -alternate { } { }
+} {{} {} {}}
+
+test csv-91.14-3 {split empty fields} {
+ csv::split -alternate { 0 2} { }
+} {{} 0 {} 2}
+
+test csv-91.15-3 {split empty fields} {
+ csv::split -alternate {1 3 } { }
+} {1 {} 3 {}}
+
+
+test csv-91.8-4 {split empty fields} {
+ csv::split -alternate {1,2,"",""}
+} {1 2 {} {}}
+
+test csv-91.9-4 {split empty fields} {
+ csv::split -alternate {1,2,3,""}
+} {1 2 3 {}}
+
+test csv-91.10-4 {split empty fields} {
+ csv::split -alternate {"","",1,2}
+} {{} {} 1 2}
+
+test csv-91.11-4 {split empty fields} {
+ csv::split -alternate {"",0,1,2}
+} {{} 0 1 2}
+
+test csv-91.12-4 {split empty fields} {
+ csv::split -alternate {"",""}
+} {{} {}}
+
+test csv-91.13-4 {split empty fields} {
+ csv::split -alternate {"","",""}
+} {{} {} {}}
+
+test csv-91.14-4 {split empty fields} {
+ csv::split -alternate {"",0,"",2}
+} {{} 0 {} 2}
+
+test csv-91.15-4 {split empty fields} {
+ csv::split -alternate {1,"",3,""}
+} {1 {} 3 {}}
+
+
+test csv-92.0.1 {split} {
+ csv::split {"xxx",yyy}
+} {xxx yyy}
+
+test csv-92.0.2 {split} {
+ csv::split -alternate {"xxx",yyy}
+} {xxx yyy}
+
+test csv-92.1.1 {split} {
+ csv::split {"xx""x",yyy}
+} {xx\"x yyy}
+
+test csv-92.1.2 {split} {
+ csv::split -alternate {"xx""x",yyy}
+} {xx\"x yyy}
+
+# -------------------------------------------------------------------------
+
+
+test csv-100.1 {custom delimiter, split} {
+ csv::split $str1_ , =
+} {123 =a= {} hello}
+
+test csv-100.2 {custom delimiter, split} {
+ csv::split $str2_ , =
+} {1 { o, =a= ,b } { 3}}
+
+test csv-100.3 {custom delimiter, split} {
+ csv::split $str3_ , =
+} {1 { o, =,= ,b } { 3}}
+
+test csv-100.4 {custom delimiter, split} {
+ csv::split $str4_ , =
+} {1 { foo,bar,baz} { 3}}
+
+test csv-100.5 {custom delimiter, split} {
+ csv::split $str5_ , =
+} {1 ==a== b}
+
+test csv-100.6 {custom delimiter, split} {
+ csv::split $str6_ , =
+} {123 123,521.2 {Mary says "Hello, I am Mary"}}
+
+test csv-100.7 {custom delimiter, split on join} {
+ # csv 0.1 was exposed to the RE \A matching problem with regsub -all
+ set x [list "\"hello, you\"" a b c]
+ ::csv::split [::csv::join $x , =] , =
+} [list "\"hello, you\"" a b c]
+
+test csv-100.8-1 {custom delimiter, split empty fields} {
+ csv::split {1 2 == ==} { } =
+} {1 2 = =}
+
+test csv-100.9-1 {custom delimiter, split empty fields} {
+ csv::split {1 2 3 ==} { } =
+} {1 2 3 =}
+
+test csv-100.10-1 {custom delimiter, split empty fields} {
+ csv::split {== == 1 2} { } =
+} {= = 1 2}
+
+test csv-100.11-1 {custom delimiter, split empty fields} {
+ csv::split {== 0 1 2} { } =
+} {= 0 1 2}
+
+test csv-100.12-1 {custom delimiter, split empty fields} {
+ csv::split {== ==} { } =
+} {= =}
+
+test csv-100.13-1 {custom delimiter, split empty fields} {
+ csv::split {== == ==} { } =
+} {= = =}
+
+test csv-100.14-1 {custom delimiter, split empty fields} {
+ csv::split {== 0 == 2} { } =
+} {= 0 = 2}
+
+test csv-100.15-1 {custom delimiter, split empty fields} {
+ csv::split {1 == 3 ==} { } =
+} {1 = 3 =}
+
+test csv-100.8-2 {custom delimiter, split empty fields} {
+ csv::split "1,2,,"
+} {1 2 {} {}}
+
+test csv-100.9-2 {custom delimiter, split empty fields} {
+ csv::split "1,2,3,"
+} {1 2 3 {}}
+
+test csv-100.10-2 {custom delimiter, split empty fields} {
+ csv::split ",,1,2"
+} {{} {} 1 2}
+
+test csv-100.11-2 {custom delimiter, split empty fields} {
+ csv::split ",0,1,2"
+} {{} 0 1 2}
+
+test csv-100.12-2 {custom delimiter, split empty fields} {
+ csv::split ","
+} {{} {}}
+
+test csv-100.13-2 {custom delimiter, split empty fields} {
+ csv::split ",,"
+} {{} {} {}}
+
+test csv-100.14-2 {custom delimiter, split empty fields} {
+ csv::split ",0,,2"
+} {{} 0 {} 2}
+
+test csv-100.15-2 {custom delimiter, split empty fields} {
+ csv::split "1,,3,"
+} {1 {} 3 {}}
+
+test csv-100.8-3 {custom delimiter, split empty fields} {
+ csv::split {1 2 } { } =
+} {1 2 {} {}}
+
+test csv-100.9-3 {custom delimiter, split empty fields} {
+ csv::split {1 2 3 } { } =
+} {1 2 3 {}}
+
+test csv-100.10-3 {custom delimiter, split empty fields} {
+ csv::split { 1 2} { } =
+} {{} {} 1 2}
+
+test csv-100.11-3 {custom delimiter, split empty fields} {
+ csv::split { 0 1 2} { } =
+} {{} 0 1 2}
+
+test csv-100.12-3 {custom delimiter, split empty fields} {
+ csv::split { } { } =
+} {{} {}}
+
+test csv-100.13-3 {custom delimiter, split empty fields} {
+ csv::split { } { } =
+} {{} {} {}}
+
+test csv-100.14-3 {custom delimiter, split empty fields} {
+ csv::split { 0 2} { } =
+} {{} 0 {} 2}
+
+test csv-100.15-3 {custom delimiter, split empty fields} {
+ csv::split {1 3 } { } =
+} {1 {} 3 {}}
+
+test csv-100.8-4 {custom delimiter, split empty fields} {
+ csv::split {1,2,==,==} , =
+} {1 2 = =}
+
+test csv-100.9-4 {custom delimiter, split empty fields} {
+ csv::split {1,2,3,==} , =
+} {1 2 3 =}
+
+test csv-100.10-4 {custom delimiter, split empty fields} {
+ csv::split {==,==,1,2} , =
+} {= = 1 2}
+
+test csv-100.11-4 {custom delimiter, split empty fields} {
+ csv::split {==,0,1,2} , =
+} {= 0 1 2}
+
+test csv-100.12-4 {custom delimiter, split empty fields} {
+ csv::split {==,==} , =
+} {= =}
+
+test csv-100.13-4 {custom delimiter, split empty fields} {
+ csv::split {==,==,==} , =
+} {= = =}
+
+test csv-100.14-4 {custom delimiter, split empty fields} {
+ csv::split {==,0,==,2} , =
+} {= 0 = 2}
+
+test csv-100.15-4 {custom delimiter, split empty fields} {
+ csv::split {1,==,3,==} , =
+} {1 = 3 =}
+
+# Try various separator characters
+
+foreach {n sep} {
+ 0 | 1 + 2 *
+ 3 / 4 \ 5 [
+ 6 ] 7 ( 8 )
+ 9 ? 10 , 11 ;
+ 12 . 13 - 14 @
+ 15 :
+} {
+ test csv-100.16-$n "split on $sep" {
+ ::csv::split [join [list REC DPI AD1 AD2 AD3] $sep] $sep =
+ } {REC DPI AD1 AD2 AD3}
+}
+
+test csv-200.0 {splitting to queue, bad separator, empty} {
+ ::struct::queue q
+ catch {::csv::split2queue q dummy-line {}} result
+ q destroy
+ set result
+} {illegal separator character "", is empty}
+
+test csv-200.1 {splitting to queue, bad separator, string} {
+ ::struct::queue q
+ catch {::csv::split2queue q dummy-line foo} result
+ q destroy
+ set result
+} {illegal separator character "foo", is a string}
+
+test csv-200.2 {splitting to matrix, bad separator, empty} {
+ ::struct::matrix m
+ catch {::csv::split2matrix m dummy-line {}} result
+ m destroy
+ set result
+} {illegal separator character "", is empty}
+
+test csv-200.3 {splitting to matrix, bad separator, string} {
+ ::struct::matrix m
+ catch {::csv::split2matrix m dummy-line foo} result
+ m destroy
+ set result
+} {illegal separator character "foo", is a string}
+
+
+testsuiteCleanup
+return
diff --git a/tcllib/modules/csv/eval.csv b/tcllib/modules/csv/eval.csv
new file mode 100644
index 0000000..f8b8988
--- /dev/null
+++ b/tcllib/modules/csv/eval.csv
@@ -0,0 +1,6 @@
+023,EVAL cmd eval in list obj var,26,45,57.78
+024,EVAL cmd eval as list,23,42,54.76
+025,EVAL cmd eval as string,53,92,57.61
+026,EVAL cmd and mixed lists,3805,11276,33.74
+027,EVAL list cmd and mixed lists,3812,11325,33.66
+028,EVAL list cmd and pure lists,592,1598,37.05
diff --git a/tcllib/modules/csv/mem_debug_bench.csv b/tcllib/modules/csv/mem_debug_bench.csv
new file mode 100644
index 0000000..281ccaf
--- /dev/null
+++ b/tcllib/modules/csv/mem_debug_bench.csv
@@ -0,0 +1,251 @@
+000,VERSIONS:,2:8.4a3,1:8.4a3,1:8.4a3%
+001,CATCH return ok,7,13,53.85
+002,CATCH return error,68,91,74.73
+003,CATCH no catch used,7,14,50.00
+004,IF if true numeric,12,33,36.36
+005,IF elseif true numeric,15,47,31.91
+006,IF else true numeric,15,46,32.61
+007,IF if true num/num,13,32,40.62
+008,IF if false num/num,13,32,40.62
+009,IF if false al/num,28,57,49.12
+010,IF if true al/num,34,54,62.96
+011,IF if false al/num,34,58,58.62
+012,IF if true al/al,33,100,33.00
+013,IF elseif true al/al,50,87,57.47
+014,IF else true al/al,50,92,54.35
+015,SWITCH first true,50,81,61.73
+016,SWITCH second true,55,84,65.48
+017,SWITCH ninth true,56,96,58.33
+018,SWITCH default true,48,81,59.26
+019,DATA create in a list,5419,13514,40.10
+020,DATA create in an array,5861,15537,37.72
+021,DATA access in a list,4424,9967,44.39
+022,DATA access in an array,4373,9167,47.70
+023,EVAL cmd eval in list obj var,26,45,57.78
+024,EVAL cmd eval as list,23,42,54.76
+025,EVAL cmd eval as string,53,92,57.61
+026,EVAL cmd and mixed lists,3805,11276,33.74
+027,EVAL list cmd and mixed lists,3812,11325,33.66
+028,EVAL list cmd and pure lists,592,1598,37.05
+029,EXPR unbraced,174,250,69.60
+030,EXPR braced,27,60,45.00
+031,EXPR inline,28,51,54.90
+032,EXPR one operand,8,13,61.54
+033,EXPR ten operands,15,25,60.00
+034,EXPR fifty operands,46,73,63.01
+035,EXPR incr with incr,13,20,65.00
+036,EXPR incr with expr,8,14,57.14
+037,KLIST shuffle0 llength 1,154,260,59.23
+038,KLIST shuffle0 llength 10,521,950,54.84
+039,KLIST shuffle0 llength 100,4126,7781,53.03
+040,KLIST shuffle0 llength 1000,46309,85434,54.20
+041,KLIST shuffle0 llength 10000,612676,1000055,61.26
+042,KLIST shuffle1 llength 1,100,181,55.25
+043,KLIST shuffle1 llength 10,432,835,51.74
+044,KLIST shuffle1 llength 100,5872,14144,41.52
+045,KLIST shuffle1 llength 1000,1293956,1235661,104.72
+046,KLIST shuffle1a llength 1,115,200,57.50
+047,KLIST shuffle1a llength 10,442,1012,43.68
+048,KLIST shuffle1a llength 100,4212,9609,43.83
+049,KLIST shuffle1a llength 1000,42350,98262,43.10
+050,KLIST shuffle1a llength 10000,445084,1052460,42.29
+051,KLIST shuffle2 llength 1,123,205,60.00
+052,KLIST shuffle2 llength 10,484,922,52.49
+053,KLIST shuffle2 llength 100,4377,8347,52.44
+054,KLIST shuffle2 llength 1000,46002,89585,51.35
+055,KLIST shuffle2 llength 10000,525442,926369,56.72
+056,KLIST shuffle3 llength 1,116,196,59.18
+057,KLIST shuffle3 llength 10,420,911,46.10
+058,KLIST shuffle3 llength 100,3730,8465,44.06
+059,KLIST shuffle3 llength 1000,39397,87416,45.07
+060,KLIST shuffle3 llength 10000,949689,1391544,68.25
+061,KLIST shuffle4 llength 1,116,204,56.86
+062,KLIST shuffle4 llength 10,450,1000,45.00
+063,KLIST shuffle4 llength 100,4067,9326,43.61
+064,KLIST shuffle4 llength 1000,39142,92580,42.28
+065,KLIST shuffle4 llength 10000,421581,944205,44.65
+066,"STR/LIST length; obj shimmer",3268,6767,48.29
+067,"LIST length; pure list",17,21,80.95
+068,STR length of a LIST,12,25,48.00
+069,"LIST exact search; first item",18,24,75.00
+070,"LIST exact search; middle item",74,111,66.67
+071,"LIST exact search; last item",142,236,60.17
+072,"LIST exact search; non-item",344,603,57.05
+073,"LIST sorted search; first item",19,29,65.52
+074,"LIST sorted search; middle item",19,27,70.37
+075,"LIST sorted search; last item",19,27,70.37
+076,"LIST sorted search; non-item",19,27,70.37
+077,"LIST exact search; untyped item",148,230,64.35
+078,"LIST exact search; typed item",107,119,89.92
+079,"LIST sorted search; typed item",18,29,62.07
+080,LIST sort,3620,4994,72.49
+081,LIST typed sort,2923,3885,75.24
+082,LIST remove first element,310,763,40.63
+083,LIST remove middle element,308,761,40.47
+084,LIST remove last element,312,757,41.22
+085,LIST replace first element,291,740,39.32
+086,LIST replace middle element,295,741,39.81
+087,LIST replace last element,295,743,39.70
+088,LIST replace first el with multiple,315,770,40.91
+089,LIST replace middle el with multiple,314,764,41.10
+090,LIST replace last el with multiple,288,750,38.40
+091,LIST replace range,288,737,39.08
+092,LIST remove in mixed list,411,959,42.86
+093,LIST replace in mixed list,398,932,42.70
+094,LIST index first element,14,24,58.33
+095,LIST index middle element,14,28,50.00
+096,LIST index last element,14,28,50.00
+097,LIST insert an item at start,297,750,39.60
+098,LIST insert an item at middle,303,746,40.62
+099,"LIST insert an item at ""end""",299,746,40.08
+100,"LIST small; early range",26,41,63.41
+101,"LIST small; late range",23,33,69.70
+102,"LIST large; early range",42,94,44.68
+103,"LIST large; late range",41,106,38.68
+104,LIST append to list,406,426,95.31
+105,LIST join list,1147,1687,67.99
+106,"LOOP for; iterate list",6848,16393,41.77
+107,"LOOP foreach; iterate list",2169,5913,36.68
+108,LOOP for (to 1000),2756,8183,33.68
+109,LOOP while (to 1000),2753,8181,33.65
+110,"LOOP for; iterate string",8350,15966,52.30
+111,"LOOP foreach; iterate string",2684,7094,37.83
+112,MAP string 1 val,686,1097,62.53
+113,MAP string 2 val,1578,2375,66.44
+114,MAP string 3 val,1938,2674,72.48
+115,MAP string 4 val,2427,3324,73.01
+116,MAP string 1 val -nocase,3772,5524,68.28
+117,MAP string 2 val -nocase,6633,9624,68.92
+118,MAP string 3 val -nocase,8809,12682,69.46
+119,MAP string 4 val -nocase,10692,15353,69.64
+120,MAP regsub 1 val,3884,4345,89.39
+121,MAP regsub 2 val,16420,17435,94.18
+122,MAP regsub 3 val,22056,23287,94.71
+123,MAP regsub 4 val,27550,29333,93.92
+124,MAP regsub 1 val -nocase,4004,4322,92.64
+125,MAP regsub 2 val -nocase,16519,17289,95.55
+126,MAP regsub 3 val -nocase,22075,23427,94.23
+127,MAP regsub 4 val -nocase,27981,29438,95.05
+128,"MAP string; no match",1011,1734,58.30
+129,"MAP string -nocase; no match",7090,10589,66.96
+130,"MAP regsub; no match",1226,2328,52.66
+131,"MAP regsub -nocase; no match",1287,2295,56.08
+132,MAP string short,44,58,75.86
+133,MAP regsub short,188,219,85.84
+134,MTHD direct ns proc call,8,15,53.33
+135,MTHD imported ns proc call,8,16,50.00
+136,MTHD interp alias proc call,25,44,56.82
+137,MTHD indirect proc eval,36,58,62.07
+138,MTHD indirect proc eval #2,58,100,58.00
+139,MTHD array stored proc call,11,25,44.00
+140,MTHD switch method call,53,86,61.63
+141,MTHD ns lookup call,113,189,59.79
+142,MTHD inline call,3,9,33.33
+143,PROC explicit return,7,12,58.33
+144,PROC implicit return,7,17,41.18
+145,PROC explicit return (2),7,13,53.85
+146,PROC implicit return (2),7,15,46.67
+147,PROC explicit return (3),7,12,58.33
+148,PROC implicit return (3),7,12,58.33
+149,PROC heavily commented,7,12,58.33
+150,"PROC do-nothing; no args",6,11,54.55
+151,"PROC do-nothing; one arg",7,12,58.33
+152,PROC local links with global,1611,2827,56.99
+153,PROC local links with upvar,1308,2630,49.73
+154,PROC local links with variable,1309,2358,55.51
+155,"READ 595K; gets",386913,551429,70.17
+156,"READ 595K; read",85889,164758,52.13
+157,"READ 595K; read & size",86171,164854,52.27
+158,"READ 3050b; gets",2152,3481,61.82
+159,"READ 3050b; read",561,682,82.26
+160,"READ 3050b; read & size",606,738,82.11
+161,"BREAD 595K; gets",392519,568992,68.98
+162,"BREAD 595K; read",51133,110961,46.08
+163,"BREAD 595K; read & size",51194,110552,46.31
+164,"BREAD 3050b; gets",2213,3174,69.72
+165,"BREAD 3050b; read",329,472,69.70
+166,"BREAD 3050b; read & size",377,517,72.92
+167,REGEXP literal regexp,48,58,82.76
+168,REGEXP var-based regexp,51,60,85.00
+169,REGEXP count all matches,149,161,92.55
+170,REGEXP extract all matches,201,255,78.82
+171,STARTUP time to launch tclsh,26402,32329,81.67
+172,STR str [string compare],15,38,39.47
+173,STR str [string equal],15,38,39.47
+174,"STR str $a equal """"",13,32,40.62
+175,"STR str num == """"",15,38,39.47
+176,STR str $a eq $b,21,49,42.86
+177,STR str $a ne $b,21,49,42.86
+178,STR str $a eq $b (same obj),19,45,42.22
+179,STR str $a ne $b (same obj),19,46,41.30
+180,STR length (==4010),13,23,56.52
+181,STR index 0,19,30,63.33
+182,STR index 100,20,31,64.52
+183,STR index 500,19,30,63.33
+184,STR index2 0,20,32,62.50
+185,STR index2 100,21,30,70.00
+186,STR index2 500,20,31,64.52
+187,STR first (success),17,23,73.91
+188,STR first (failure),115,116,99.14
+189,STR first (total failure),106,103,102.91
+190,STR last (success),17,23,73.91
+191,STR last (failure),91,109,83.49
+192,STR last (total failure),82,86,95.35
+193,"STR match; simple (success early)",17,31,54.84
+194,"STR match; simple (success late)",18,30,60.00
+195,"STR match; simple (failure)",18,28,64.29
+196,"STR match; simple (total failure)",16,29,55.17
+197,"STR match; complex (success early)",18,34,52.94
+198,"STR match; complex (success late)",152,165,92.12
+199,"STR match; complex (failure)",121,134,90.30
+200,"STR match; complex (total failure)",95,101,94.06
+201,"STR range; index 100..200 of 4010",26,40,65.00
+202,"STR replace; no replacement",87,126,69.05
+203,"STR replace; equal replacement",93,133,69.92
+204,"STR replace; longer replacement",103,146,70.55
+205,"STR repeat; abcdefghij * 10",16,23,69.57
+206,"STR repeat; abcdefghij * 100",48,47,102.13
+207,"STR repeat; abcdefghij * 1000",231,257,89.88
+208,"STR repeat; 4010 chars * 10",282,744,37.90
+209,"STR repeat; 4010 chars * 100",6976,14673,47.54
+210,"STR reverse iter1; 100 chars",1534,2295,66.84
+211,"STR reverse iter1; 100 uchars",1457,2322,62.75
+212,"STR reverse iter2; 100 chars",1123,2042,55.00
+213,"STR reverse iter2; 100 uchars",1042,1972,52.84
+214,"STR reverse recur1; 100 chars",3458,7067,48.93
+215,"STR reverse recur1; 100 uchars",3523,6650,52.98
+216,"STR split; 4010 chars",2806,4605,60.93
+217,"STR split; 12100 uchars",7890,13813,57.12
+218,"STR split iter; 4010 chars",11129,28087,39.62
+219,"STR split iter; 12100 uchars",33318,86314,38.60
+220,STR append,99,160,61.88
+221,STR append (1KB + 1KB),95,134,70.90
+222,STR append (10KB + 1KB),209,537,38.92
+223,STR append (1MB + 2b * 1000),38681,190529,20.30
+224,STR append (1MB + 1KB),28344,173073,16.38
+225,STR append (1MB + 1KB * 20),29077,173622,16.75
+226,STR append (1MB + 1KB * 1000),66893,207868,32.18
+227,STR append (1MB + 1MB * 3),125505,327765,38.29
+228,STR append (1MB + 1MB * 5),158507,855295,18.53
+229,STR append (1MB + (1b + 1K + 1b) * 100),33101,174031,19.02
+230,STR info locals match,946,1521,62.20
+231,TRACE no trace set,34,121,28.10
+232,TRACE read,34,50,68.00
+233,TRACE write,33,50,66.00
+234,TRACE unset,33,48,68.75
+235,TRACE all set (rwu),34,52,65.38
+236,UNSET var exists,12,19,63.16
+237,UNSET catch var exists,13,23,56.52
+238,UNSET catch var !exist,77,105,73.33
+239,UNSET info check var exists,16,27,59.26
+240,UNSET info check var !exist,12,27,44.44
+241,UNSET nocomplain var exists,12,18,66.67
+242,UNSET nocomplain var !exist,12,16,75.00
+243,VAR access locally set,10,19,52.63
+244,VAR access local proc arg,10,20,50.00
+245,VAR access global,35,49,71.43
+246,VAR access upvar,40,54,74.07
+247,VAR set scalar,7,15,46.67
+248,VAR set array element,14,28,50.00
+249,VAR 100 'set's in array,161,272,59.19
+250,VAR 'array set' of 100 elems,306,467,65.52
diff --git a/tcllib/modules/csv/mem_debug_bench_a.csv b/tcllib/modules/csv/mem_debug_bench_a.csv
new file mode 100644
index 0000000..8e17485
--- /dev/null
+++ b/tcllib/modules/csv/mem_debug_bench_a.csv
@@ -0,0 +1,256 @@
+000,VERSIONS:,2:8.4a3,1:8.4a3,1:8.4a3%
+001,CATCH return ok,7,13,53.85
+002,CATCH return error,68,91,74.73
+003,CATCH no catch used,7,14,50.00
+004,IF if true numeric,12,33,36.36
+005,IF elseif true numeric,15,47,31.91
+
+006,IF else true numeric,15,46,32.61
+007,IF if true num/num,13,32,40.62
+008,IF if false num/num,13,32,40.62
+009,IF if false al/num,28,57,49.12
+010,IF if true al/num,34,54,62.96
+011,IF if false al/num,34,58,58.62
+012,IF if true al/al,33,100,33.00
+013,IF elseif true al/al,50,87,57.47
+014,IF else true al/al,50,92,54.35
+015,SWITCH first true,50,81,61.73
+016,SWITCH second true,55,84,65.48
+017,SWITCH ninth true,56,96,58.33
+018,SWITCH default true,48,81,59.26
+019,DATA create in a list,5419,13514,40.10
+020,DATA create in an array,5861,15537,37.72
+021,DATA access in a list,4424,9967,44.39
+
+022,DATA access in an array,4373,9167,47.70
+023,EVAL cmd eval in list obj var,26,45,57.78
+024,EVAL cmd eval as list,23,42,54.76
+025,EVAL cmd eval as string,53,92,57.61
+026,EVAL cmd and mixed lists,3805,11276,33.74
+027,EVAL list cmd and mixed lists,3812,11325,33.66
+028,EVAL list cmd and pure lists,592,1598,37.05
+029,EXPR unbraced,174,250,69.60
+030,EXPR braced,27,60,45.00
+031,EXPR inline,28,51,54.90
+032,EXPR one operand,8,13,61.54
+033,EXPR ten operands,15,25,60.00
+
+
+034,EXPR fifty operands,46,73,63.01
+035,EXPR incr with incr,13,20,65.00
+036,EXPR incr with expr,8,14,57.14
+037,KLIST shuffle0 llength 1,154,260,59.23
+038,KLIST shuffle0 llength 10,521,950,54.84
+039,KLIST shuffle0 llength 100,4126,7781,53.03
+040,KLIST shuffle0 llength 1000,46309,85434,54.20
+041,KLIST shuffle0 llength 10000,612676,1000055,61.26
+042,KLIST shuffle1 llength 1,100,181,55.25
+043,KLIST shuffle1 llength 10,432,835,51.74
+044,KLIST shuffle1 llength 100,5872,14144,41.52
+045,KLIST shuffle1 llength 1000,1293956,1235661,104.72
+046,KLIST shuffle1a llength 1,115,200,57.50
+047,KLIST shuffle1a llength 10,442,1012,43.68
+048,KLIST shuffle1a llength 100,4212,9609,43.83
+049,KLIST shuffle1a llength 1000,42350,98262,43.10
+050,KLIST shuffle1a llength 10000,445084,1052460,42.29
+051,KLIST shuffle2 llength 1,123,205,60.00
+052,KLIST shuffle2 llength 10,484,922,52.49
+
+053,KLIST shuffle2 llength 100,4377,8347,52.44
+054,KLIST shuffle2 llength 1000,46002,89585,51.35
+055,KLIST shuffle2 llength 10000,525442,926369,56.72
+056,KLIST shuffle3 llength 1,116,196,59.18
+057,KLIST shuffle3 llength 10,420,911,46.10
+058,KLIST shuffle3 llength 100,3730,8465,44.06
+059,KLIST shuffle3 llength 1000,39397,87416,45.07
+060,KLIST shuffle3 llength 10000,949689,1391544,68.25
+061,KLIST shuffle4 llength 1,116,204,56.86
+062,KLIST shuffle4 llength 10,450,1000,45.00
+063,KLIST shuffle4 llength 100,4067,9326,43.61
+064,KLIST shuffle4 llength 1000,39142,92580,42.28
+065,KLIST shuffle4 llength 10000,421581,944205,44.65
+066,"STR/LIST length; obj shimmer",3268,6767,48.29
+067,"LIST length; pure list",17,21,80.95
+068,STR length of a LIST,12,25,48.00
+069,"LIST exact search; first item",18,24,75.00
+070,"LIST exact search; middle item",74,111,66.67
+071,"LIST exact search; last item",142,236,60.17
+072,"LIST exact search; non-item",344,603,57.05
+073,"LIST sorted search; first item",19,29,65.52
+074,"LIST sorted search; middle item",19,27,70.37
+075,"LIST sorted search; last item",19,27,70.37
+076,"LIST sorted search; non-item",19,27,70.37
+077,"LIST exact search; untyped item",148,230,64.35
+078,"LIST exact search; typed item",107,119,89.92
+079,"LIST sorted search; typed item",18,29,62.07
+080,LIST sort,3620,4994,72.49
+081,LIST typed sort,2923,3885,75.24
+082,LIST remove first element,310,763,40.63
+083,LIST remove middle element,308,761,40.47
+084,LIST remove last element,312,757,41.22
+085,LIST replace first element,291,740,39.32
+086,LIST replace middle element,295,741,39.81
+087,LIST replace last element,295,743,39.70
+088,LIST replace first el with multiple,315,770,40.91
+089,LIST replace middle el with multiple,314,764,41.10
+090,LIST replace last el with multiple,288,750,38.40
+091,LIST replace range,288,737,39.08
+092,LIST remove in mixed list,411,959,42.86
+093,LIST replace in mixed list,398,932,42.70
+094,LIST index first element,14,24,58.33
+095,LIST index middle element,14,28,50.00
+096,LIST index last element,14,28,50.00
+097,LIST insert an item at start,297,750,39.60
+098,LIST insert an item at middle,303,746,40.62
+099,"LIST insert an item at ""end""",299,746,40.08
+100,"LIST small; early range",26,41,63.41
+101,"LIST small; late range",23,33,69.70
+102,"LIST large; early range",42,94,44.68
+103,"LIST large; late range",41,106,38.68
+104,LIST append to list,406,426,95.31
+105,LIST join list,1147,1687,67.99
+106,"LOOP for; iterate list",6848,16393,41.77
+107,"LOOP foreach; iterate list",2169,5913,36.68
+108,LOOP for (to 1000),2756,8183,33.68
+109,LOOP while (to 1000),2753,8181,33.65
+110,"LOOP for; iterate string",8350,15966,52.30
+111,"LOOP foreach; iterate string",2684,7094,37.83
+112,MAP string 1 val,686,1097,62.53
+113,MAP string 2 val,1578,2375,66.44
+114,MAP string 3 val,1938,2674,72.48
+115,MAP string 4 val,2427,3324,73.01
+116,MAP string 1 val -nocase,3772,5524,68.28
+117,MAP string 2 val -nocase,6633,9624,68.92
+118,MAP string 3 val -nocase,8809,12682,69.46
+119,MAP string 4 val -nocase,10692,15353,69.64
+120,MAP regsub 1 val,3884,4345,89.39
+121,MAP regsub 2 val,16420,17435,94.18
+122,MAP regsub 3 val,22056,23287,94.71
+123,MAP regsub 4 val,27550,29333,93.92
+124,MAP regsub 1 val -nocase,4004,4322,92.64
+125,MAP regsub 2 val -nocase,16519,17289,95.55
+126,MAP regsub 3 val -nocase,22075,23427,94.23
+127,MAP regsub 4 val -nocase,27981,29438,95.05
+128,"MAP string; no match",1011,1734,58.30
+129,"MAP string -nocase; no match",7090,10589,66.96
+130,"MAP regsub; no match",1226,2328,52.66
+131,"MAP regsub -nocase; no match",1287,2295,56.08
+132,MAP string short,44,58,75.86
+133,MAP regsub short,188,219,85.84
+134,MTHD direct ns proc call,8,15,53.33
+135,MTHD imported ns proc call,8,16,50.00
+136,MTHD interp alias proc call,25,44,56.82
+137,MTHD indirect proc eval,36,58,62.07
+138,MTHD indirect proc eval #2,58,100,58.00
+139,MTHD array stored proc call,11,25,44.00
+140,MTHD switch method call,53,86,61.63
+141,MTHD ns lookup call,113,189,59.79
+142,MTHD inline call,3,9,33.33
+143,PROC explicit return,7,12,58.33
+144,PROC implicit return,7,17,41.18
+145,PROC explicit return (2),7,13,53.85
+146,PROC implicit return (2),7,15,46.67
+147,PROC explicit return (3),7,12,58.33
+148,PROC implicit return (3),7,12,58.33
+149,PROC heavily commented,7,12,58.33
+150,"PROC do-nothing; no args",6,11,54.55
+151,"PROC do-nothing; one arg",7,12,58.33
+152,PROC local links with global,1611,2827,56.99
+153,PROC local links with upvar,1308,2630,49.73
+154,PROC local links with variable,1309,2358,55.51
+155,"READ 595K; gets",386913,551429,70.17
+156,"READ 595K; read",85889,164758,52.13
+157,"READ 595K; read & size",86171,164854,52.27
+158,"READ 3050b; gets",2152,3481,61.82
+159,"READ 3050b; read",561,682,82.26
+160,"READ 3050b; read & size",606,738,82.11
+161,"BREAD 595K; gets",392519,568992,68.98
+162,"BREAD 595K; read",51133,110961,46.08
+163,"BREAD 595K; read & size",51194,110552,46.31
+164,"BREAD 3050b; gets",2213,3174,69.72
+165,"BREAD 3050b; read",329,472,69.70
+166,"BREAD 3050b; read & size",377,517,72.92
+167,REGEXP literal regexp,48,58,82.76
+168,REGEXP var-based regexp,51,60,85.00
+169,REGEXP count all matches,149,161,92.55
+170,REGEXP extract all matches,201,255,78.82
+171,STARTUP time to launch tclsh,26402,32329,81.67
+172,STR str [string compare],15,38,39.47
+173,STR str [string equal],15,38,39.47
+174,"STR str $a equal """"",13,32,40.62
+175,"STR str num == """"",15,38,39.47
+176,STR str $a eq $b,21,49,42.86
+177,STR str $a ne $b,21,49,42.86
+178,STR str $a eq $b (same obj),19,45,42.22
+179,STR str $a ne $b (same obj),19,46,41.30
+180,STR length (==4010),13,23,56.52
+181,STR index 0,19,30,63.33
+182,STR index 100,20,31,64.52
+183,STR index 500,19,30,63.33
+184,STR index2 0,20,32,62.50
+185,STR index2 100,21,30,70.00
+186,STR index2 500,20,31,64.52
+187,STR first (success),17,23,73.91
+188,STR first (failure),115,116,99.14
+189,STR first (total failure),106,103,102.91
+190,STR last (success),17,23,73.91
+191,STR last (failure),91,109,83.49
+192,STR last (total failure),82,86,95.35
+193,"STR match; simple (success early)",17,31,54.84
+194,"STR match; simple (success late)",18,30,60.00
+195,"STR match; simple (failure)",18,28,64.29
+196,"STR match; simple (total failure)",16,29,55.17
+197,"STR match; complex (success early)",18,34,52.94
+198,"STR match; complex (success late)",152,165,92.12
+199,"STR match; complex (failure)",121,134,90.30
+200,"STR match; complex (total failure)",95,101,94.06
+201,"STR range; index 100..200 of 4010",26,40,65.00
+202,"STR replace; no replacement",87,126,69.05
+203,"STR replace; equal replacement",93,133,69.92
+204,"STR replace; longer replacement",103,146,70.55
+205,"STR repeat; abcdefghij * 10",16,23,69.57
+206,"STR repeat; abcdefghij * 100",48,47,102.13
+207,"STR repeat; abcdefghij * 1000",231,257,89.88
+208,"STR repeat; 4010 chars * 10",282,744,37.90
+209,"STR repeat; 4010 chars * 100",6976,14673,47.54
+210,"STR reverse iter1; 100 chars",1534,2295,66.84
+211,"STR reverse iter1; 100 uchars",1457,2322,62.75
+212,"STR reverse iter2; 100 chars",1123,2042,55.00
+213,"STR reverse iter2; 100 uchars",1042,1972,52.84
+214,"STR reverse recur1; 100 chars",3458,7067,48.93
+215,"STR reverse recur1; 100 uchars",3523,6650,52.98
+216,"STR split; 4010 chars",2806,4605,60.93
+217,"STR split; 12100 uchars",7890,13813,57.12
+218,"STR split iter; 4010 chars",11129,28087,39.62
+219,"STR split iter; 12100 uchars",33318,86314,38.60
+220,STR append,99,160,61.88
+221,STR append (1KB + 1KB),95,134,70.90
+222,STR append (10KB + 1KB),209,537,38.92
+223,STR append (1MB + 2b * 1000),38681,190529,20.30
+224,STR append (1MB + 1KB),28344,173073,16.38
+225,STR append (1MB + 1KB * 20),29077,173622,16.75
+226,STR append (1MB + 1KB * 1000),66893,207868,32.18
+227,STR append (1MB + 1MB * 3),125505,327765,38.29
+228,STR append (1MB + 1MB * 5),158507,855295,18.53
+229,STR append (1MB + (1b + 1K + 1b) * 100),33101,174031,19.02
+230,STR info locals match,946,1521,62.20
+231,TRACE no trace set,34,121,28.10
+232,TRACE read,34,50,68.00
+233,TRACE write,33,50,66.00
+234,TRACE unset,33,48,68.75
+235,TRACE all set (rwu),34,52,65.38
+236,UNSET var exists,12,19,63.16
+237,UNSET catch var exists,13,23,56.52
+238,UNSET catch var !exist,77,105,73.33
+239,UNSET info check var exists,16,27,59.26
+240,UNSET info check var !exist,12,27,44.44
+241,UNSET nocomplain var exists,12,18,66.67
+242,UNSET nocomplain var !exist,12,16,75.00
+243,VAR access locally set,10,19,52.63
+244,VAR access local proc arg,10,20,50.00
+245,VAR access global,35,49,71.43
+246,VAR access upvar,40,54,74.07
+247,VAR set scalar,7,15,46.67
+248,VAR set array element,14,28,50.00
+249,VAR 100 'set's in array,161,272,59.19
+250,VAR 'array set' of 100 elems,306,467,65.52
diff --git a/tcllib/modules/csv/pkgIndex.tcl b/tcllib/modules/csv/pkgIndex.tcl
new file mode 100644
index 0000000..538e735
--- /dev/null
+++ b/tcllib/modules/csv/pkgIndex.tcl
@@ -0,0 +1,2 @@
+if {![package vsatisfies [package provide Tcl] 8.4]} {return}
+package ifneeded csv 0.8.1 [list source [file join $dir csv.tcl]]