For the most part this library works great! But I'm encountering some strange things when iterating over a parsed CSV in my unit tests. Sometimes the parsing seems to fail and the values are populated with goggly goop like this column value of \0\0\0è•ä\u{05}\0\0\0\0\0\0\0\0\0\0\0\0\0…+\u{07}\u{01}\0\0\0Ø.

Not sure if anyone else has encounter this, I haven't been able to figure out any sort of patter as to why it is happening.

I would love to submit a fix, @yaslab please let me know if I can go ahead.

Also-- I see you have some automated tests. I've never run automated tests in Xcode, but I was thinking I can figure it out and add a contributing.md file for this repo to document how to run and add tests for future features or bugfixes. Might make it easier for others to open up good pull requests and share the workload.

This code causes a memory leak.

struct DecodableExample: Decodable {
    let dateKey: Date
    let stringKey: String
    let optionalStringKey: String?
}


for i in 1...20 {
    var records = [DecodableExample]()
    var ii = 0
    do {
        let csv = try String(contentsOfFile: "/tmp/test.txt", encoding: .utf8)
        let reader = try CSVReader(string: csv, hasHeaderRow: true, trimFields: true, delimiter: ";", whitespaces: .whitespaces)
        let decoder = CSVRowDecoder()
        let formatter = DateFormatter()
        formatter.dateFormat = "yyyy-MM-dd HH:mm:ss"
        decoder.dateDecodingStrategy = .formatted(formatter)
        while reader.next() != nil {
            ii += 1
            let row = try decoder.decode(DecodableExample.self, from: reader)
            records.append(row)
        }
    } catch {
        print("\(ii)-\(error)")
    }
    print(i)
}

file : test.txt

memory graph:

causes

[!] CocoaPods could not find compatible versions for pod "CSV.swift": In Podfile: CSV.swift (~> 2.2.1)

None of your spec sources contain a spec satisfying the dependency: CSV.swift (~> 2.2.1).

Sometimes, CSV files are formatted in a way that they stay well human readable, for example with a delimiter character followed by a tabstop. Would it be possible to integrate a functionality to ignore, for example, all \t in the input stream? I know that I can preprocess my CSV files to achieve the same, but this cannot always be ensured for other users of the software.

This code causes a memory leak. If I use JSON decoder then there is no memory leak. Any idea how to fix this?

import Foundation
import CSV

typealias  TestCodable = Test & Codable

protocol Test {
    func baz()
}

struct Product1:TestCodable {
    var title:String
    var price:Int
    var quantity:Int
    func baz() {
        print(title)
    }
}

let csv = """
title,price,quantity
Nike,10,1
Nike1,11,1
Nike2,12,1
Nike3,13,1
Nike4,13,1
Nike5,13,1
Nike6,13,1
"""

let json = """
{
"title": "Nike",
"price": 10,
"quantity": 1
}
"""
let data = json.data(using: .utf8)!
var arrayProduct: [Test] = []
let reader = try CSVReader(string: csv, hasHeaderRow: true)
let decoder = CSVRowDecoder()

while reader.next() != nil {
//    let product: Product1 = try! JSONDecoder().decode(Product1.self, from: data)
    let product = try! decoder.decode(Product1.self, from: reader)
    arrayProduct.append(product)
    print("\(product)")
    sleep(4)
}

This code causes a memory leak.

struct DecodableExample: Decodable {
    let dateKey: Date
    let stringKey: String
    let optionalStringKey: String?
}
[test.txt](https://github.com/yaslab/CSV.swift/files/2708834/test.txt)


for i in 1...20 {
    var records = [DecodableExample]()
    var ii = 0
    do {
        let csv = try String(contentsOfFile: "/tmp/test.txt", encoding: .utf8)
        let reader = try CSVReader(string: csv, hasHeaderRow: true, trimFields: true, delimiter: ";", whitespaces: .whitespaces)
        let decoder = CSVRowDecoder()
        let formatter = DateFormatter()
        formatter.dateFormat = "yyyy-MM-dd HH:mm:ss"
        decoder.dateDecodingStrategy = .formatted(formatter)
        while reader.next() != nil {
            ii += 1
            let row = try decoder.decode(DecodableExample.self, from: reader)
            records.append(row)
        }
    } catch {
        print("\(ii)-\(error)")
    }
    print(i)
}

Developer may now opt to implement Decodable using Int-based CodingKey, instead of String. This is particularly useful if the data does not have headers, or if the header strings are unwieldy.

Example Decodable implementation:

struct IntKeyedDecodableExample: Decodable, Equatable {
        private enum CodingKeys: Int, CodingKey {
            case intKey = 2
            case stringKey = 0
            case optionalStringKey = 1
            case dateKey = 4
            case enumKey = 5
        }
        
        let intKey: Int
        let stringKey: String
        let optionalStringKey: String?
        let dateKey: Date
        let enumKey: Enum
        
        func toRow() -> String {
            return "\(self.stringKey),\(self.optionalStringKey ?? ""),\(self.intKey),,\"\(CSVReader.dateFormatter.string(from: self.dateKey))\",\(self.enumKey)\n"
        }
        
        static func ==(left: IntKeyedDecodableExample, right: IntKeyedDecodableExample) -> Bool {
            let formatter = CSVReader.dateFormatter
            return left.intKey == right.intKey && left.stringKey == right.stringKey && left.optionalStringKey == right.optionalStringKey
                //&& left.dateKey.compare(right.dateKey) == ComparisonResult.orderedSame // TODO: find more accurate conversion method, cannot compare directly likely because we are losing precision when in csv
                && formatter.string(from: left.dateKey) == formatter.string(from: right.dateKey)
                && left.enumKey == right.enumKey
        }
        
        static var examples: [IntKeyedDecodableExample] {
            return [
                IntKeyedDecodableExample(intKey: 12345, stringKey: "stringValue", optionalStringKey: nil, dateKey: Date(), enumKey: .first),
                IntKeyedDecodableExample(intKey: 54321, stringKey: "stringValue2", optionalStringKey: "withValue", dateKey: Date(timeInterval: 100, since: Date()), enumKey: .second)
            ]
        }
    }

I've been working to add support & tests for Int typed CodingKey, for supporting Decodable with or without headers. Can I claim this feature if you're not already on it?

Expected Behavior:

• When implementing Decodable for a type, developer can use Int type enum for CodingKey
• Value of the enum maps to column of the data
• Header row should be ignored
• Developer continues to use Configuration struct to indicate the presence/absence of header row

Example:

    class IntKeyedDecodableExample: SupportedDecodableExample {
        private enum CodingKeys: Int, CodingKey {
            case intKey = 2
            case stringKey = 0
            case optionalStringKey = 1
            case dateKey = 4
            case enumKey = 5
        }
    }

Improved speed of reading from file. (6a18924, 028bd2f)

Result of the performance test

• OS: Ubuntu 16.04
• Swift: 4.2.1-RELEASE
• Build Configuration: release
• CSV file info
  • File size: 13.3MB (15310 records, 55 fields)
  • Encoding: UTF-8 (Including ASCII and Japanese)

| | Before (v2.2.1) | After (this branch) | Delta | | ---- | ---- | ---- | ---- | | 1st | 35.313s | 5.811s | -83.5% | | 2nd | 35.964s | 5.746s | -84.0% | | 3rd | 35.672s | 5.732s | -83.9% |

Test code

// main.swift
import Foundation
import CSV

let stream = InputStream(fileAtPath: "/path/to/file.csv")!
let csv = try! CSVReader(stream: stream, hasHeaderRow: true)
while csv.next() != nil {}

Any way to speed this up with GCD? I have a CSV with 50k+ rows that I need to crunch through and just incrementing a counter takes about 45 seconds on my iPad.

// Time began:
let startTime = Date()

// Read the CSV file:
let filePath = Bundle.main.path(forResource: "foo", ofType: "csv")
let stream = InputStream(fileAtPath: filePath!)
let csv = try! CSVReader(stream: stream!, hasHeaderRow: true)

// Count the rows:
var counter = 0
while csv.next() != nil {
    counter += 1
}

// Check the results:
print("Rows: \(counter)")
print("Elapsed time: \(startTime.timeIntervalSinceNow) seconds")

Output:
Rows: 52229
Elapsed time: -46.1442109942436 seconds

Ran into this issue when parsing a CSV file that contained JSON-like data with newlines. Example:

"{(
    2fa,
    \"Apple Watch\"
)}","Another column","A third column"

From what it looks like, it gets split it into 3 rows.

If you, however, replace the double quotes with single quotes, it works just fine:

"Tags","Type","third"
"{(
    2fa,
    \'Apple Watch\'
)}","Another column","A third column"

I know, it's a very specific edge case, but I just wanted to let you know

Hi, thanks to your great effort. I really think the code is straightforward to use, especially for Swift beginners. I have successfully create a new CSV file via this method, and I'm curious about how to upload the file to the cloud? eg. google doc, or create a website to download? In addition, since my MacBook was produced in 2016, I just installed SPM and didn't even execute it, but it still took a long time. Is there any other simple version so that I can use the SPM more smoothly? Thanks a lot!

I am reading billions of rows of data from gzipped CSV files exported from Google Cloud Compute's BigQuery. Here is a Swift script using CSV.swift to read a gzipped CSV file with 12 columns and 848,563 rows:

import Foundation

import CSV
import Gzip

// CSV.swift
// Takes 1m5s without decoding the objects
// Takes 5m15s when decoding to objects
func testReadBigQueryCSV() {
    let filePath = "data/my-data-export-2020-09-20-000000000161.csv.gz"
    let fileData: Data = try! Data(contentsOf: URL(fileURLWithPath: filePath))
    let decodedBody: Data = try! fileData.gunzipped()
    let stream = InputStream(data: decodedBody)
    let csv = try! CSVReader(stream: stream, hasHeaderRow: true)
    print(csv.headerRow!)
    var rows: [[String]] = []
    let decoder = CSVRowDecoder()
    decoder.userInfo[.knownFormatKey] = KnownFormat.BigQueryCSV
    for row in csv {
        rows.append(row)
    }
    /*var rows: [MyDecodableType] = []*/
    /*while csv.next() != nil {*/
        /*print("\(row)")*/
        /*let row: MyDecodableType = try! decoder.decode(MyDecodableType.self, from: csv)*/
        /*rows.append(row)*/
    /*}*/
    print("Got \(rows.count) rows")
}

When simply reading the CSV fields into an [[String]], it takes 1m5s. When decoding the Strings into types, it takes 5m15s.

For comparison, here is a Python script that reads and parses the same file in 3.6 seconds:

#!/usr/bin/env python3

import pandas as pd

# Takes 3.6 seconds when not parsing the dates
# Takes 2m43s when parsing the dates
def main():
    file_path = "data/my-data-export-2020-09-20-000000000161.csv.gz"
    print("Reading {}".format(file_path))
    # df = pd.read_csv(file_path, compression='gzip', parse_dates=["time", "timeUpdateReceived", "inserted"])
    df = pd.read_csv(file_path, compression='gzip')
    print("Read {} rows".format(df.shape))
    print(df.columns)
    print(df.dtypes)
    print(df.iloc[0])

if __name__ == "__main__":
    main()

The comparison is 3.6s vs 1m5s and 2m43s vs 5m15s. That's an 18x slower read. pandas also uses a single CPU core.

If there is a space between HTML division tags, From the white space It consider that as separate row. I have attached screenshot of my sample data for reference. Could you please help me in this? I just want to know if there are specific formats needs to be followed for HTML texts in CSV.

Sorry if this is too simple but I'm quite new to Swift so this is all unfamiliar to me. I pasted this code into xcode and got the error above:

import Foundation
import CSV

let stream = OutputStream(toFileAtPath: "file.csv", append: false)!
let csv = try! CSVWriter(stream: stream)

try! csv.write(row: ["id", "name"])
try! csv.write(row: ["1", "foo"])
try! csv.write(row: ["1", "bar"])

csv.stream.close()

Particularly it was pointing at this line: let csv = try! CSVWriter(stream: stream)

In what seems to be the console, I got this:

Fatal error: 'try!' expression unexpectedly raised an error: CSV.CSVError.cannotOpenFile: file /BuildRoot/Library/Caches/com.apple.xbs/Sources/swiftlang_Fall2018/swiftlang_Fall2018-1000.11.42/src/swift/stdlib/public/core/ErrorType.swift, line 184