Ss-json - High-performance json parsing in swift

I have a JSON line file and I'm trying to parse it, I'm wondering if this can be used?

Currently swift-json is ~4 times slower than the Foundation implementation, but I suspect this is because I am recreating the decoder each time:

swift-json (~97 seconds)

let (bytes, urlResponse) = try! await URLSession.shared.bytes(from: file)
var logs: [Log] = []
do {
    let startTime = CFAbsoluteTimeGetCurrent()
    for try await line in bytes.lines {
        let trimmed = line.trimmingCharacters(in: .whitespaces)
        if !trimmed.isEmpty {
            let decoder: JSON = try Grammar.parse(trimmed.utf8, as: JSON.Rule<String.Index>.Root.self)
            let log: Log = try .init(from: decoder)
            logs.append(log)
        }
    }
    let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
    print("Time elapsed: \(timeElapsed) s.")
} catch {
    print("Reading file failed with error \(error)")
}

Foundation (~24 seconds)

let decoder: JSONDecoder = JSONDecoder()
let (bytes, urlResponse) = try! await URLSession.shared.bytes(from: file)
var logs: [Log] = []
do {
    let startTime = CFAbsoluteTimeGetCurrent()
    for try await line in bytes.lines {
        let data = line.trimmingCharacters(in: .whitespaces).data(using: .utf8) ?? Data()
        if !data.isEmpty {
            let log: Log = try! decoder.decode(Log.self, from: data)
            logs.append(log)
        }
    }
    let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
    print("Time elapsed: \(timeElapsed) s.")
} catch {
    print("Reading file failed with error \(error)")
}

Tested on a ~100MB file with 140000 lines

the existing API isn’t very clear about whether the passed keys are explicitly allowed to appear, or explicitly prohibited from appearing. we should give it an argument label, like whitelisting:

feedback on current API is that is is too complex and expects users to interact with swift-grammar types. this API exists to support high-performance use-cases, but many other use cases just want to parse a String and dispatch to Decodable.

in particular, we should have

let _:SomeDecodable = try .init(from: try JSON.init(parsing: #"{ "x": 1 }"#))

benchmarks CI caught a major performance regression with commit 59d34c5f8c337b068d07ad5b0e4caa2fd0b8176e

before:

 [15/16] Linking benchmark
Build complete! (50.69s)
swift-json: decoded 38295 messages
foundation: decoded 38295 messages
swift-json decoding time: 0.497709959 seconds
foundation decoding time: 4.90599839 seconds

after:

[15/16] Linking benchmark
Build complete! (45.21s)
swift-json: decoded 38295 messages
foundation: decoded 38295 messages
swift-json decoding time: 3.21054002 seconds
foundation decoding time: 4.539217732 seconds

xcodebuild crashes if there exists a directory named _Snippets, and for some reason the SPM has not implemented snippetsDirectory, so we must disable snippets for now.

this means the snippets have lost CI coverage, so this is only a short-term solution.

the v0.3.0 API should greatly reduce the likelihood of users needing to interact with the Grammar module, so we should stop importing it with @_exported.

_move has broken again, due to compiler drift:

error: Can not use feature when experimental move only is disabled! Pass the frontend flag -enable-experimental-move-only to swift to enable the usage of this language feature
                return try body(_move value)

we should reevaluate usage of _move in this API, since it likely serves no purpose.

adds API taking closure arguments to LintingDictionary and [JSON] that records debugging information if decoding fails, and removes unnecessary closure arguments elsewhere

struct Decimal:Codable  
{
    let units:Int 
    let places:Int 
}

I'm trying to wrap my head around the need for having a user-defined Decimal type for decoding, as opposed to leveraging Swift's own Decimal type. It's rather odd to me, a user of the library, having introduce an arbitrary abstraction to be able to decode something into a basic numeric type.

It would be great, if this library could eventually support JSON5, which comes with a bundle of amazing features.

Foundation on macOS supports it by now (through .json5Allowed), but it looks like swift-corelibs-foundation doesn't (yet?).