Plugin and runtime library for using protobuf with Swift

Overview

Swift logo

Swift Protobuf

Welcome to Swift Protobuf!

Apple's Swift programming language is a perfect complement to Google's Protocol Buffer ("protobuf") serialization technology. They both emphasize high performance and programmer safety.

This project provides both the command-line program that adds Swift code generation to Google's protoc and the runtime library that is necessary for using the generated code. After using the protoc plugin to generate Swift code from your .proto files, you will need to add this library to your project.

Build and Test Check Upstream Protos Run Conformance Tests

Features of SwiftProtobuf

SwiftProtobuf offers many advantages over alternative serialization systems:

  • Safety: The protobuf code-generation system avoids the errors that are common with hand-built serialization code.
  • Correctness: SwiftProtobuf passes both its own extensive test suite and Google's full conformance test for protobuf correctness.
  • Schema-driven: Defining your data structures in a separate .proto schema file clearly documents your communications conventions.
  • Idiomatic: SwiftProtobuf takes full advantage of the Swift language. In particular, all generated types provide full Swift copy-on-write value semantics.
  • Efficient binary serialization: The .serializedData() method returns a Data with a compact binary form of your data. You can deserialize the data using the init(serializedData:) initializer.
  • Standard JSON serialization: The .jsonUTF8Data() method returns a JSON form of your data that can be parsed with the init(jsonUTF8Data:) initializer.
  • Hashable, Equatable: The generated struct can be put into a Set<> or Dictionary<>.
  • Performant: The binary and JSON serializers have been extensively optimized.
  • Extensible: You can add your own Swift extensions to any of the generated types.

Best of all, you can take the same .proto file and generate Java, C++, Python, or Objective-C for use on other platforms. The generated code for those languages will use the exact same serialization and deserialization conventions as SwiftProtobuf, making it easy to exchange serialized data in binary or JSON forms, with no additional effort on your part.

Documentation

More information is available in the associated documentation:

  • Google's protobuf documentation provides general information about protocol buffers, the protoc compiler, and how to use protocol buffers with C++, Java, and other languages.
  • PLUGIN.md documents the protoc-gen-swift plugin that adds Swift support to the protoc program
  • API.md documents how to use the generated code. This is recommended reading for anyone using SwiftProtobuf in their project.
  • cocoadocs.org has the generated API documentation
  • INTERNALS.md documents the internal structure of the generated code and the library. This should only be needed by folks interested in working on SwiftProtobuf itself.
  • STYLE_GUIDELINES.md documents the style guidelines we have adopted in our codebase if you are interested in contributing

Getting Started

If you've worked with Protocol Buffers before, adding Swift support is very simple: you just need to build the protoc-gen-swift program and copy it into your PATH. The protoc program will find and use it automatically, allowing you to build Swift sources for your proto files. You will also, of course, need to add the SwiftProtobuf runtime library to your project as explained below.

System Requirements

To use Swift with Protocol buffers, you'll need:

  • A Swift 4.2 or later compiler (Xcode 10.0 or later). Support is included for the Swift Package Manager; or using the included Xcode project. The Swift protobuf project is being developed and tested against the latest release version of Swift available from Swift.org

  • Google's protoc compiler. The Swift protoc plugin is being actively developed and tested against the latest protobuf sources. The SwiftProtobuf tests need a version of protoc which supports the swift_prefix option (introduced in protoc 3.2.0). It may work with earlier versions of protoc. You can get recent versions from Google's github repository.

Building and Installing the Code Generator Plugin

To translate .proto files into Swift, you will need both Google's protoc compiler and the SwiftProtobuf code generator plugin.

Building the plugin should be simple on any supported Swift platform:

$ git clone https://github.com/apple/swift-protobuf.git
$ cd swift-protobuf

Pick what released version of SwiftProtobuf you are going to use. You can get a list of tags with:

$ git tag -l

Once you pick the version you will use, set your local state to match, and build the protoc plugin:

$ git checkout tags/[tag_name]
$ swift build -c release

This will create a binary called protoc-gen-swift in the .build/release directory.

To install, just copy this one executable into a directory that is part of your PATH environment variable.

NOTE: The Swift runtime support is now included with macOS. If you are using old Xcode versions or are on older system versions, you might need to use also use --static-swift-stdlib with swift build.

Alternatively install via Homebrew

If you prefer using Homebrew:

$ brew install swift-protobuf

This will install protoc compiler and Swift code generator plugin.

Converting .proto files into Swift

To generate Swift output for your .proto files, you run the protoc command as usual, using the --swift_out=<directory> option:

$ protoc --swift_out=. my.proto

The protoc program will automatically look for protoc-gen-swift in your PATH and use it.

Each .proto input file will get translated to a corresponding .pb.swift file in the output directory.

More information about building and using protoc-gen-swift can be found in the detailed Plugin documentation.

Adding the SwiftProtobuf library to your project...

To use the generated code, you need to include the SwiftProtobuf library module in your project. How you do this will vary depending on how you're building your project. Note that in all cases, we strongly recommend that you use the version of the SwiftProtobuf library that corresponds to the version of protoc-gen-swift you used to generate the code.

...using swift build

After copying the .pb.swift files into your project, you will need to add the SwiftProtobuf library to your project to support the generated code. If you are using the Swift Package Manager, add a dependency to your Package.swift file and import the SwiftProtobuf library into the desired targets. Adjust the "1.6.0" here to match the [tag_name] you used to build the plugin above:

dependencies: [
    .package(name: "SwiftProtobuf", url: "https://github.com/apple/swift-protobuf.git", from: "1.6.0"),
],
targets: [
    .target(name: "MyTarget", dependencies: ["SwiftProtobuf"]),
]

...using Xcode

If you are using Xcode, then you should:

  • Add the .pb.swift source files generated from your protos directly to your project
  • Add the appropriate SwiftProtobuf_<platform> target from the Xcode project in this package to your project.

...using CocoaPods

If you're using CocoaPods, add this to your Podfile adjusting the :tag to match the [tag_name] you used to build the plugin above:

pod 'SwiftProtobuf', '~> 1.0'

And run pod install.

NOTE: CocoaPods 1.7 or newer is required.

...using Carthage

If you're using Carthage, add this to your Cartfile but adjust the tag to match the [tag_name] you used to build the plugin above:

github "apple/swift-protobuf" ~> 1.0

Run carthage update and drag SwiftProtobuf.framework into your Xcode.project.

Quick Start

Once you have installed the code generator, used it to generate Swift code from your .proto file, and added the SwiftProtobuf library to your project, you can just use the generated types as you would any other Swift struct.

For example, you might start with the following very simple proto file:

syntax = "proto3";

message BookInfo {
   int64 id = 1;
   string title = 2;
   string author = 3;
}

Then generate Swift code using:

$ protoc --swift_out=. DataModel.proto

The generated code will expose a Swift property for each of the proto fields as well as a selection of serialization and deserialization capabilities:

// Create a BookInfo object and populate it:
var info = BookInfo()
info.id = 1734
info.title = "Really Interesting Book"
info.author = "Jane Smith"

// As above, but generating a read-only value:
let info2 = BookInfo.with {
    $0.id = 1735
    $0.title = "Even More Interesting"
    $0.author = "Jane Q. Smith"
  }

// Serialize to binary protobuf format:
let binaryData: Data = try info.serializedData()

// Deserialize a received Data object from `binaryData`
let decodedInfo = try BookInfo(serializedData: binaryData)

// Serialize to JSON format as a Data object
let jsonData: Data = try info.jsonUTF8Data()

// Deserialize from JSON format from `jsonData`
let receivedFromJSON = try BookInfo(jsonUTF8Data: jsonData)

You can find more information in the detailed API Documentation.

Report any issues

If you run into problems, please send us a detailed report. At a minimum, please include:

  • The specific operating system and version (for example, "macOS 10.12.1" or "Ubuntu 16.10")
  • The version of Swift you have installed (from swift --version)
  • The version of the protoc compiler you are working with from protoc --version
  • The specific version of this source code (you can use git log -1 to get the latest commit ID)
  • Any local changes you may have
Comments
  • Problems with generated sources in multiple modules with Swift 3.1

    Problems with generated sources in multiple modules with Swift 3.1

    I upgraded to Swift 3.1 Then I updated to swift-protobuf 0.9.29 Compiled proto-gen-swift from the 0.9.29 sources and use the 0.9.29 version as a dependency.

    Some protobuf files that were previously working correctly now generate hundreds of errors like:

    /.../Work/Swift/Systems/****/.build/checkouts/***Protobuf-5523697989699535816/Sources/***Messages.pb.swift:1292:22: note: overloads for '!=' exist with these partially matching parameter lists: (Any.Type?, Any.Type?), (UInt8, UInt8), (Int8, Int8), (UInt16, UInt16), (Int16, Int16), (UInt32, UInt32), (Int32, Int32), (UInt64, UInt64), (Int64, Int64), (UInt, UInt), (Int, Int), (ContiguousArray<Element>, ContiguousArray<Element>), (ArraySlice<Element>, ArraySlice<Element>), (T?, T?), (T?, _OptionalNilComparisonType), (_OptionalNilComparisonType, T?), ((A, B), (A, B)), ((A, B, C), (A, B, C)), ((A, B, C, D), (A, B, C, D)), ((A, B, C, D, E), (A, B, C, D, E)), ((A, B, C, D, E, F), (A, B, C, D, E, F)), (LazyFilterIndex<Base>, LazyFilterIndex<Base>), ([Key : Value], [Key : Value])
        if snapshotEntry != other.snapshotEntry {return false}
                         ^
    /.../Work/Swift/Systems/****/.build/checkouts/***-5523697989699535816/Sources/***Messages.pb.swift:1476:17: error: binary operator '!=' cannot be applied to two '[Proto****]' operands
        if trendbar != other.trendbar {return false}
           ~~~~~~~~ ^  ~~~~~~~~~~~~~~
    

    Any help appreciated!

    swift version: 3.1 protoc version: 3.2.0 protoc-gen-swift version: 0.9.29 MacOS: 10.12.4

    help wanted 
    opened by gmosx 31
  • EXC_BAD_ACCESS in decodeMessage(decoder:) due to stack overflow

    EXC_BAD_ACCESS in decodeMessage(decoder:) due to stack overflow

    Note: This is likely a duplicate of https://github.com/apple/swift-protobuf/issues/1014 but I didn't want to cram all of this into a comment in a long chain. Feel free to close as duplicate if you want.

    Summary/TLDR

    The change to move computed properties backed by _storage to properties stored directly on the message structs has caused massive size increases for messages at the top of a deep hierarchy. The switch code generated for oneof parsing is sensitive to these size increases, leading to massive stack frames. In debug mode without optimization these stack frames can easily exhaust the 512KB stack of a background dispatch thread and cause a crash.

    Details

    We are experiencing a crash in debug builds after upgrading SwiftProtobuf and protoc-gen-swift from 1.7 to 1.10.2. Our crash happens in the decodeMessage(decoder:) method of a message containing a oneof with lots of cases. From what I can tell the stack frame generated for the withExtendedLifetime closure in that method is huge, about 800KB. Since the decoding is happening on a background thread with a 512KB stack, the app crashes as soon as this closure is invoked.

    The message is of the form:

    message RenderDataDTO {
      SomeMessage1 m1 = 1;
      SomeMessage2 m2 = 2;
      SomeMessage3 m3 = 3;
      oneof render {
        SomeCaseMessage1 case1 = 100;
        SomeCaseMessage2 case2 = 101;
        SomeCaseMessage3 case3 = 102;
        // 20 cases in total
        SomeCaseMessage20 case20 = 120;
      }
    

    The code that gets generated for decodeMessage(decoder:) has the form:

      public mutating func decodeMessage<D: SwiftProtobuf.Decoder>(decoder: inout D) throws {
        _ = _uniqueStorage()    // LINE A
        try withExtendedLifetime(_storage) { (_storage: _StorageClass) in    // LINE B
          while let fieldNumber = try decoder.nextFieldNumber() {
            switch fieldNumber {
            case 1: try decoder.decodeSingularMessageField(value: &_storage._m1)
            case 2: try decoder.decodeSingularMessageField(value: &_storage._m2)
            case 3: try decoder.decodeSingularStringField(value: &_storage._m3)
            case 100:
              var v: SomeCaseMessage1?
              if let current = _storage._render {
                try decoder.handleConflictingOneOf()
                if case .case1(let m) = current {v = m}
              }
              try decoder.decodeSingularMessageField(value: &v)
              if let v = v {_storage._case1 = .case1(v)}
            case 101:
              var v: SomeCaseMessage2?
              if let current = _storage._render {
                try decoder.handleConflictingOneOf()
                if case .case2(let m) = current {v = m}
              }
              try decoder.decodeSingularMessageField(value: &v)
              if let v = v {_storage._case2 = .case2(v)}
           // etc...
    

    If I set a breakpoint at the line marked LINE A and check the stack pointer, then set a breakpoint at LINE B and do the same I find that the difference is about 800KB.

    Here's one example:

    RenderDataDTO.decodeMessage(decoder:)
           rsp = 0x00007000087cdb60
    
    In closure:
           rsp = 0x0000700008704780   <-- 800KB jump!
    

    I have a couple of observations to explain this:

    Observation 1:

    In the earlier version of swift-protobuf almost all fields of Message structs were stored in _storage, a reference-typed container, and exposed as computed vars. In the current version they seem to much more often (but not always?) be implemented as properties on the message itself. Thus the size of a leaf message struct with no child messages became O(n) instead of O(1), where n is the number of fields in the message.

    The story gets a lot worse when you have child messages. Since messages are structs, children get embedded directly in their parents, so the size of a non-leaf message grows like the sum of its entire message sub-tree. So the size scales more like O(n^m), where n is the average number of children per level and m is the average number of levels in the message hierarchy. Since Swift always likes to allocate structs on the stack, this can lead to massive stack frames.

    This agrees with what I saw when I measured the size of the SomeCaseMessageN messages using MemoryLayout<SomeCaseMessageN>.size. In the older code, they were all 24 bytes, with a few exceptions. The largest was 80 bytes. In the new code the sizes tended to increase by 10X-20X. 224 and 448 bytes were typical sizes, with several messages growing above 1000B. One grew to 5256 bytes, while the biggest was 6080 bytes.

    Observation 2:

    If the compiler is smart, it will know that all the cases in the switch statement inside that closure are mutually independent. Only one of them will ever execute per invocation, so their local variables can share space on the stack, and the size of the stack frame will be the size of the biggest case in the switch. Since we're in debug mode it might not be making that optimization, and instead allocating separate space for every case in the switch. Given the size impact of the changes I discussed above, this could add up to a lot of space.

    In fact, when I change the optimization mode from none to size or speed, the message sizes don't change but the size of the stack frame drops to about 26KB. That's still an oversized stack frame eating 5% of the stack in release mode, but at least it's not bigger than the entire stack!

    For reference, with the code generated from 1.7, the stack frame is about 16KB with optimization mode none. With optimization mode speed the stack frame drops to just 400B.

    Wrapping up

    There seem to be problems with putting so many stored properties on messages. Perhaps this could be dialed back to where it was before?

    Environment: OS: MacOS Catalina 10.15.6 Xcode: Version 11.6 Swift 5

    opened by n8gray 28
  • Issue parsing unknown enum case

    Issue parsing unknown enum case

    iOS 13.4 Xcode Version 11.4 (11E146) Swift 5 Swift-Protobuf 1.8.0 proto2

    My iOS app is using generated code from outdated proto files. Since I've compiled the .swift files from the old protos, we've added some new cases to one of our enums, and started returning the cases to the client from the backend. I'm getting an error when parsing the JSON data into the Swift Proto objects: unrecognizedEnumValue.

    From my understanding of the proto spec, it should be fine to add new enum cases, and clients who don't have the latest spec map the unknown value to the 0 case. Is there something wrong with how we name our 0th case?

    Old Proto

    message Product {
        enum State {
            State_INVALID = 0;
            ACTIVE = 1;
            DISCONTINUED = 2;
        }
        optional State state = 1;
    }
    

    New Proto

    message Product {
        enum State {
            State_INVALID = 0;
            ACTIVE = 1;
            DISCONTINUED = 2;
            FOR_SALE = 3;
            NOT_FOR_SALE = 4;
        }
        optional State state = 1;
    }
    

    JSON Being Parsed

    {
        "state": "FOR_SALE"
    }
    

    Swift JSON Decoding (using Old Proto)

    do {
        var options = JSONDecodingOptions()
        options.ignoreUnknownFields = true
        return try Product(jsonUTF8Data: data, options: options)
    } catch {
        print(error) // error is printed unrecognizedEnumValue
        return nil
    }
    
    opened by mpdifran 27
  • Conform to CustomDebugStringConvertible for Debug only

    Conform to CustomDebugStringConvertible for Debug only

    Resolves #1245

    This boxes all references to CustomDebugStringConvertible and debugDescription in #if checks to exclude them from the release binary. The existence of this conformance and default implementation on the Message protocol (and some other types) adds up quite considerably. Across our large iOS app, we can attribute 6.7 MB of the uncompressed binary size to generated swift protos. This change cuts that back by 780 KB (11.6%). Other efforts like #1240 will probably also help trim this back even further.

    I tried to do this with as little diff as possible. Added some inline comments on some things I'm unsure about.

    Special thanks to @bubski for the idea.

    v2.0 
    opened by dflems 22
  • SwiftProtobuf plugin for SwiftPM?

    SwiftProtobuf plugin for SwiftPM?

    tl;dr: Is it possible for this repo to offer a canonical SwiftProtobuf plugin for SwiftPM, as demonstrated here?

    Now that Swift 5.6 is out in the Xcode 13.3 betas and later, Swift Packages now support build tool plugins, as defined in SE-0303 and SE-0325.

    However, for the time being, it seems like everyone will have to roll their own protobuf to Swift generator plugin. It doesn't seem like there is a plugin already "out there" and one does not simply copy and paste the example from the proposal. Speaking for myself, I am not that familiar with the tool and I would certainly have no idea how to support a hand rolled plugin going forward.

    Is it possible for this repo to offer a canoncial SwiftProtobuf plugin for SwiftPM? All I really want to do is declare a dependency to this repo and add a plug in to my target. Is this straightforward usability naive?

    Some details about my use case. I just have some protobuf files organised into folders that I want to generate Swift from. I want to compile that source into a library. This library is a dependency among other targets in different packages.

    macOS Monterey 12.2 (21D49) Xcode 13.3 / Build version 13E5086k swift-driver version: 1.44.2 Apple Swift version 5.6 (swiftlang-5.6.0.320.8 clang-1316.0.18.8) Target: x86_64-apple-macosx12.0 https://github.com/apple/swift-protobuf/releases/tag/1.18.0

    opened by kielgillard 21
  • add github actions workflow for build and test on linux

    add github actions workflow for build and test on linux

    Use the official swift docker images to execute make build test on pull requests targeting master. Creates a test matrix for swift and protobuf versions, currently running tests on both swift 5.1.5 and 5.2 with protobuf 3.11.4. The sub-targets of the test target are broken into steps to allow easier diagnostics and caching of the conformance test runner.

    I tried to get caching of the conformance-test-runner working, but I haven't seen it work yet and I'm not sure why.

    To see this working: https://github.com/toffaletti/swift-protobuf/pull/1/checks

    opened by toffaletti 21
  • Swift 5: adopt new Data.withUnsafeBytes API

    Swift 5: adopt new Data.withUnsafeBytes API

    Unfortunately I wasn't able to use the #if compiler(>=5.0) flag since the project supports older swift versions where this directive is not available.

    opened by alanzeino 21
  • Add SPM plugin

    Add SPM plugin

    Motivation

    SPM is providing new capabilities for tools to expose themselves as plugins which allows them to generate build commands. This allows us to create a plugin for SwiftProtobuf which invokes the protoc compiler and generates the Swift code. Creating such a plugin is greatly wanted since it improves the usage of the protoc-gen-swift plugin dramatically. Fixes https://github.com/apple/swift-protobuf/issues/1207

    Modification

    This PR adds a new SPM plugin which generates build commands for generating Swift files from proto files. Since users of the plugin might have complex setups, I introduced a new swift-protobuf-config.json file that adopters have to put into the root of their target which wants to use the new plugin. The format of this configuration file is quite simple:

    {
        "invocations": [
            {
                "protoFiles": [
                    "Foo.proto"
                ],
                "visibility": "internal"
            }
        ]
    }
    
    

    It allows you to configure multiple invocations to the protoc compiler. For each invocation you have to pass the relative path from the target source to the proto file. Additionally, you have to decide which visibility the generated symbols should have. In my opinion, this configuration files gives us the most flexibility and more complex setups to be covered as well.

    Open topics

    Hosting of the protoc binary

    Hosting of the protoc binary is the last open thing to figure out before we can release a plugin for SwiftProtobuf. From my point of view, there are three possible routes we can take:

    1. Include the artifactbundle inside the SwiftProtobuf repository
    2. Include the artifactebundle as an artifact on the GH releases in the protobuf repo
    3. Extend the the artifact bundle manifest to allow inclusion of URLs. This would require a Swift evolution pitch most likely.

    However, with all three of the above we would still need to release a new version of SwiftProtobuf with every new release of protoc.

    Future work

    Proto dependencies between modules

    With the current shape of the PR one can already use dependencies between proto files inside a single target. However, it might be desirable to be able to build dependency chains of Swift targets where each target includes proto files which depend on protoc files from the dependencies of the Swift target. I punted this from the initial plugin because this requires a bit more work and thinking. Firstly, how would you even spell such an import? Secondly, the current way of doing ProtoPathModuleMapping files is not super ideal for this approach. It might make sense to introduce a proto option to set the Swift module name inside the proto files already.

    Result

    We now have a SPM plugin that can generate Swift code from proto files. To use it, it provides a configuration file format to declare different invocations to the protoc compiler.

    TODOs:

    • As soon as we agreed on the binary problem, I am going to write documentation how to use the plugin
    opened by FranzBusch 20
  • Could you add advanced/experimental option to ignore default initializer ( and to enable Memberwise Initializers )?

    Could you add advanced/experimental option to ignore default initializer ( and to enable Memberwise Initializers )?

    Hi! Could you add support for swift 5.1?

    By the way, how could I disable all checks for swift 4.2 in generated proto files? I would like to have only latest swift version in generated proto files.

    Thanks!

    opened by lolgear 19
  • Proposal:  Move `descriptor` to SwiftProtobuf library for general use

    Proposal: Move `descriptor` to SwiftProtobuf library for general use

    OS: macOS 10.13.3 Xcode version: 9.3 beta 4 Swift version: 4.1 SwiftProtobuf version: 1.0.3

    I previously posted this issue in the grpc-swift project (here) but MrMage referred me to the swift-protobuf project since I mistakenly assumed it was a problem with grpc-swift.

    In a *.proto file we are defining a custom method option in the following way:

    extend google.protobuf.MethodOptions {
        bool requiresAuthentication = 50001;
    }
    

    When generating the Swift code an import for SwiftProtobufPluginLibrary is missing in the generated *.pb.swift file. Google_Protobuf_MethodOptions is being extended in the generated code but it is being marked as an undeclared type until I manually import SwiftProtobufPluginLibrary in addition to the already imported SwiftProtobuf. This is fixing the issue but since ideally the generated code is never modified manually this is not a great solution.

    Is this a known issue and/or has somebody encountered this issue before?

    opened by basalphenaar 19
  • Emit Sendable conformances

    Emit Sendable conformances

    • Implicit Sendable conformance for all Message and Enum.
    • Generated Sendable conformance for oneof enums.
    • Generated @unchecked Sendable for messages with heap storage.
    • Explicit Sendable conformance for extension-related tooling.

    Notes:

    • Tested by applying the changes to a medium-sized codebase that has pervasive oneof and heap storage, but few uses of advanced features like extensions or RPC.
    • The code is formulated to be pre-5.5 compatible, while avoiding putting two copies of the entire type definition inside #if. However, I don't have an older compiler handy to check.
    • I left types and protocols used purely during serialize/deserialize alone, like ExtensionMap, _NameMap, [Blah][De|En|codingOptions, etc.
    • I didn’t bump the compatibility version since it doesn’t technically create API or ABI incompatibilities — just warnings, for now. I don’t know if this was the right call.

    Tasks:

    • [x] Investigate generating extension Blah: @unchecked Sendable {} unconditionally for messages, enums, and oneof types to avoid having to bump the API compatibility version.
    • [x] Update generated comments: https://github.com/apple/swift-protobuf/pull/1208#discussion_r801482071
    opened by zwaldowski 18
  • The 1.x branch can't build protobuf on the image for 4.2.4 Swift

    The 1.x branch can't build protobuf on the image for 4.2.4 Swift

    As seen on the Checks for #1354, upstream protobuf now needs a newer compiler than what we have on the CI images. So either we need to find a way to update CI on the 1_x_release_branch or we shut off the tests there that require building protobuf.

    opened by thomasvl 2
  • 1.x branch: Fail to encode google.protobuf.Value number if not finite.

    1.x branch: Fail to encode google.protobuf.Value number if not finite.

    From https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Value

    Note that attempting to serialize NaN or Infinity results in error. (We can't serialize these as string "NaN" or "Infinity" values like we do for regular fields, because they would parse as string_value, not number_value).

    opened by thomasvl 2
  • Release 1.20.4

    Release 1.20.4

    We would like to kindly ask if there is anything blocking the next release?

    We would like to have an official release that includes https://github.com/apple/swift-protobuf/pull/1345 that was merged on November 15th 2022.

    Apologies if there is a release tracker that I am not aware of :)

    opened by antongrbin 1
  • SwiftPM plugin: generate files into the source directory

    SwiftPM plugin: generate files into the source directory

    Hello,

    I would like to keep the generated files in the repository as well as the .proto files. This seems to be useful at least for the following reasons:

    • The protoc is not part of the project, and its version is something that is defined by the environment. I want to be sure that some strange CI environment (with old or buggy version of the protoc) will not ruin my production.
    • Check and monitor any change to the generated files (who did when and why), especially in case of protoc version change.

    The current version doesn't work for me as it puts the generated files into the .build folder.

    I would like to either:

    • be able to specify the path for the generated file, and make it possible to put it to the source path.
    • have a command to generate swift files from protobuf files.

    It seems that https://github.com/apple/swift-evolution/blob/main/proposals/0332-swiftpm-command-plugins.md allows both:

    • PluginPermission allows to write to the package directory
    • CommandPlugin allows to have a plugin command to perform any action.

    The second approach is preferable for me as it allows to run the generation on-request from the developer.

    opened by gliush 4
  • Protobuf blocking xCode indexing with weird error

    Protobuf blocking xCode indexing with weird error

    I'm using xCode 14.1 on macbook pro silicon M1 with Ventura 13.0 (22A380).

    I'm using Protobuf 1.20.2 via SPM

    I receive the following error in the xcode console while indexing.

    `Showing All Errors Only
    
    Indexing
    
    /Applications/Xcode-14.1.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -swift-version 5 -I /Applications/Xcode-14.1.0.app/Contents/PlugIns/IDESwiftPackageCore.framework/Versions/A/Frameworks/SwiftPM.framework/SharedSupport/ManifestAPI -sdk /Applications/Xcode-14.1.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.0.sdk -package-description-version 5.6.0 -parse-as-library -I /Applications/Xcode-14.1.0.app/Contents/PlugIns/IDESwiftPackageCore.framework/Versions/A/Frameworks/SwiftPM.framework/SharedSupport/PluginAPI /Users/me/Library/Developer/Xcode/DerivedData/MyProject-etyfxtttxpmavldwvibjxmvtbbhn/SourcePackages/checkouts/swift-protobuf/Plugins/SwiftProtobufPlugin/plugin.swift -index-file -index-file-path /Users/me/Library/Developer/Xcode/DerivedData/MyProject-etyfxtttxpmavldwvibjxmvtbbhn/SourcePackages/checkouts/swift-protobuf/Plugins/SwiftProtobufPlugin/plugin.swift -disable-batch-mode -o /Users/me/Library/Developer/Xcode/DerivedData/MyProject-etyfxtttxpmavldwvibjxmvtbbhn/SourcePackages/checkouts/swift-protobuf/Plugins/SwiftProtobufPlugin/plugin.swift.o -index-store-path /Users/me/Library/Developer/Xcode/DerivedData/MyProject-etyfxtttxpmavldwvibjxmvtbbhn/Index.noindex/DataStore
    
    error: module name "plugin.swift" is not a valid identifier; use -module-name flag to specify an alternate name
    error: Command failed with exit code 1`
    

    This leads to xCode not linting code, jump to definition not working and other minor indexing issues.

    Any help?

    Thank you

    opened by vx8 3
  • Removal of Data from codegen and binary deserialisation

    Removal of Data from codegen and binary deserialisation

    Note: only the last three commits are actually related to this change. Previous commits are actually part of https://github.com/apple/swift-protobuf/pull/1326, on which this PR depends.

    Motivation

    This is a follow-up PR for the removal of Data (and eventually Foundation) from swift-protobuf. This is related to https://github.com/apple/swift-protobuf/issues/816 and follows the changes made in https://github.com/apple/swift-protobuf/pull/1326

    Modifications

    This PR includes the bulk of the required changes for the removal of Data from swift-protobuf:

    • Changes to the codegen, to move away from using Data and use [UInt8] instead for byte-type fields.
    • Regenerated protos based on the codegen changes.
    • Changes to remove Data completely from the binary deserialisation path.
    • Other required changes to make things compile with the new generated code.

    Other PRs will follow, removing Data usages from the binary serialisation path, the JSON (de)serialisation, and any other miscellaneous remaining usages.

    Result

    The codegen and binary deserialisation paths are now Data-free.

    opened by gjcairo 3
Releases(1.20.3)
  • 1.20.3(Nov 8, 2022)

    • Minor Updates
      • Be more accepting for generation options. by @thomasvl in https://github.com/apple/swift-protobuf/pull/1318
      • Add the file naming option by @thomasvl in https://github.com/apple/swift-protobuf/pull/1338

    Full Changelog: https://github.com/apple/swift-protobuf/compare/1.20.2...1.20.3

    Source code(tar.gz)
    Source code(zip)
  • 1.20.2(Sep 23, 2022)

    • Minor Updates
      • For protoc generator authors (swift-grpc), bring over some descriptor improvements to aid migration. by @thomasvl in https://github.com/apple/swift-protobuf/pull/1291

    Full Changelog: https://github.com/apple/swift-protobuf/compare/1.20.1...1.20.2

    Source code(tar.gz)
    Source code(zip)
  • 1.20.1(Aug 26, 2022)

    • Minor Updates
      • Provide environment variable configuration option by @FranzBusch in https://github.com/apple/swift-protobuf/pull/1279
      • Fix warning in new package.swift by @FranzBusch in https://github.com/apple/swift-protobuf/pull/1281

    Full Changelog: https://github.com/apple/swift-protobuf/compare/1.20.0...1.20.1

    Source code(tar.gz)
    Source code(zip)
  • 1.20.0(Aug 25, 2022)

  • 1.19.1(Aug 1, 2022)

  • 1.19.0(Mar 2, 2022)

    • Major Updates
      • SwiftWasm compilable #1192
      • Sendable conformance on Messages without library changes #1208 & #1213
    • Minor Updates
      • Swift format lint issues #1199
      • Visit extensions improvement #1201 (smaller codegen)
      • Tweak generation for messages with only extension ranges. #1202 (codegen change to help compilation performance)
    Source code(tar.gz)
    Source code(zip)
  • 1.18.0(Sep 27, 2021)

    • Minor Updates
      • Support JSON ignoreUnknownFields within WKTs. #1172
      • Handle writing out json when the value is empty/zero bytes and no type. #1167
    • Fixes from fuzz testing:
      • Handle skipping a JSON object that ends after the open brace. #1165
      • When skipping a varint for an unknown field, ensure it is valid. #1169
      • Rework JSON skipping so it's not recursive for nested arrays #1178
      • Track recursion depth for nested JSON ListValue structures #1179
      • Uses same workaround for stack sizes in non optimized builds. #1183
      • Reject fieldmasks that have non-ASCII characters for JSON #1185
      • Don't overrun string when parsing timestamps #1186
    Source code(tar.gz)
    Source code(zip)
  • 1.17.0(May 14, 2021)

    • Significant Change
      • TextFormatDecodingError has a new error case to got with a recursion limit for TextFormat decoding (add safety found via fuzz testing for potential bogus input trying trigger stack overflow #1132), if you have any switch states on all the cases, this is a breaking change in that you must handle the new case.
    • Fixes from fuzz testing:
      • Fix octal TextFormat decoding failure #1124
      • Avoid walking off the end of the buffer in two parsing cases. #1126
      • Add TextFormatDecodingOptions and implement a recursion limit. #1132
      • Don't walk off the end of the buffer during a unicode bytes decode. #1136
    • Minor Updates
      • Change oneof enforcement to allow null (found via upstream conformance test requirements) #1135
      • Allow proto3_optional for extensions. #1138
      • Some edge case speed improvements:
        • Add modify operation to ExtensionFieldValueSet. #1137
        • Don't do characterwise-compares if not needed. #1145
        • Clear previous contents before decoding Any from TextFormat #1147
    Source code(tar.gz)
    Source code(zip)
  • 1.16.0(Apr 13, 2021)

    Minor Changes:

    • Normalize CRLF in comments to avoid double spacing some input. #1109
    • Fixes from fuzz testing:
      • Fix to decoding of groups/unknown fields. #1118
      • Protect against overallocation on bad binary input. #1119
      • Deal with malformed UTF8 while parsing a JSON field name. #1121
      • Avoid looping forever on malformed Map TextFormat. #1122
    Source code(tar.gz)
    Source code(zip)
  • 1.15.0(Jan 22, 2021)

    Minor Changes:

    • Allow parallel lookup of types in Any decode. #1098
    • Fix extension order serialization and improve generation for extension ranges with single values. #1100
    Source code(tar.gz)
    Source code(zip)
  • 1.14.0(Dec 2, 2020)

    Notable Changes:

    • Don't allow raw LF or CR in the middle of TextFormat string literals. #1085 – TextFormat was used as input (tests, etc.), then previously working multiline strings might no longer parse. This is to bring the library in alignment with the protocolbuffers conformance tests.

    Minor Changes:

    • Fix issue with oneof being named newValue #1087
    • Support unicode escapes in TextFormat #1085
    Source code(tar.gz)
    Source code(zip)
  • 1.13.0(Oct 23, 2020)

    Notable Changes:

    • Sort map fields for TextFormat (to match spec) #1076 Minor Changes:
    • Reduce foundation usage #1064
    • The CocoaPod spec and checked in Xcode project were updated to move the minimum iOS version to 9.0 to avoid warnings in Xcode 12 #1082
    Source code(tar.gz)
    Source code(zip)
  • 1.12.0(Aug 28, 2020)

    Notable Changes:

    • Change code generation to reduce the stack usage in non optimized builds (Issue #1034)
      • Move required field support for oneof into a generated helper (#1041)
      • Work around stack usage for non optimize build and switch statements (#1040)
      • Work around excessive stack space in non optimized builds during oneof isInitialized (#1042)
    • Revise the way storage calculations are done to take into account the cost of singular message fields since those directly increase the size of the containing message (#1046)
    • Fix JSON coding/decoding of NullValue WKT (#1051)

    Minor Changes:

    • Minor oneof tweak: don't generate case nil when also generating a default (#1035)
    • Factor out the common decodeJSON from all the wrappers (#1062)
    Source code(tar.gz)
    Source code(zip)
  • 1.11.0(Aug 6, 2020)

    Minor changes:

    • Remove empty Data singleton (#1028)
    • Factor SwiftProtobuf module name into the namer, add SwiftProtobufModuleName (#1017)
    • Eliminate NamingUtils's awareness of SwiftProtobufNamer (#1030)
    Source code(tar.gz)
    Source code(zip)
  • 1.10.2(Jul 10, 2020)

  • 1.10.1(Jul 9, 2020)

  • 1.10.0(Jun 30, 2020)

    Notable Changes:

    • Support Proto2 extensions in JSON coder/decoder (#1002)

    Minor Changes:

    • Add visitRepeated* methods to the BinaryEncodingSizeVisitory. (#1009)
    • Fix for newer Xcode/swift versions: var --> let to eliminate warning (#975)
    • Don't use StaticString.utf8Start unless there is a pointer rep. (#1015)
    Source code(tar.gz)
    Source code(zip)
  • 1.9.0(May 18, 2020)

    • Better handing of proto identifiers that start with underscores and numbers #947 #954
    • Added CMake based build for platforms with SwiftPM #957
    • Use withContiguousStorageIfAvailable for String encoding in BinaryEncoder #949
    • Make setting a repeated extension field to [] clear it #966
    • Declare the MessageExtensions with the correct visibility. #969
    • Support for new Proto3 optional (this needs a protoc from protocolbuffers/protobuf v3.12.0 (or later)) #978
    • Provide some more map helpers in the plugin library like the C++ Descriptor. #983
    • Move the SwiftProtobuf.xcodeproj build settings into xcconfig files #986
    Source code(tar.gz)
    Source code(zip)
  • 1.8.0(Jan 28, 2020)

    New features/interfaces:

    • Add Message binary decoding support from ContiguousBytes (#914)
    • Make things generic over ContiguousBytes and @inlinable (#915, #921)

    Notable changes:

    • Use heap-based storage only when a Protobuf has a singular transitive recursion (#900)
    • Use raw pointers instead of typed pointers (#918)
    • Add missing CaseIterable support for nested enums (#923)
    • Guard against TextFormat encoding of unknowns from overflowing the stack (#927)

    Performance related changes:

    • JSON/TextFormat submessage encoding performance improvements (#916)
    • Avoid collecting unknown fields while scanning for message (#924)
    • Minor code cleanup for JSON additions (#935)
    Source code(tar.gz)
    Source code(zip)
  • 1.7.0(Sep 27, 2019)

    Complete support for Swift 5.1.

    Additions to the library:

    • The generated code for descriptor.proto is now included in the library, this means one no longer has to generate and compile it in if there are proto files that had extension declared on those types. (#727)

    Notable changes:

    • The Swift 5.1 compiler will error on switch statements for an enum if it has a large number of cases; this could happen in the generated code. The generated code now splits things up to avoid this compile error. (#904)
    Source code(tar.gz)
    Source code(zip)
  • 1.6.0(Jul 18, 2019)

    Notable updates:

    • Raise a .illegalNull is the top level JSON is null. (#870)
    • Perf: Use new float formatting code (#882)
    • Specify swift_versions in podspec and minimum CocoaPods 1.7.0 version. (#891)
    Source code(tar.gz)
    Source code(zip)
  • 1.5.0(Apr 15, 2019)

    Complete support for Swift 5.

    • Swift5: Fix warnings about loss of precision (#844)
    • Drop the support for Swift <4.0. (#847)
    • Swift5.0: Change Data(bytes:) to Data(_:). (#848)
    • Fix build command to work also with Swift 5. (#852)
    • Swift 5.0: Switch a stray Data(bytes:) to Data(_:). (#854)
    • Swift 5: adopt new Data.withUnsafeBytes API. (#843)

    Update some internals:

    • Update some FileIo calls to FileHandle. (#845)
    • Remove printToFd logic from FileIo. (#846)

    Add some features to the library:

    • TextFormat conformance: Add option to suppress unknown field printing. (#850)
    • Adding JSON encoding option: Use proto field names instead of lowerCamelCase names. (#856)
    Source code(tar.gz)
    Source code(zip)
  • 1.4.0(Mar 11, 2019)

    Minor updates around some edge conditions in TextFormat/JSON format:

    • The empty string is a valid JSON encoding for a FieldMask. (#835)
    • Accept too-large float values in Text format (#840)
    Source code(tar.gz)
    Source code(zip)
  • 1.3.1(Jan 7, 2019)

  • 1.3.0(Jan 4, 2019)

    • SwiftPM Updates to be in better shape for Swift 5. (#819, #824)
    • Code updates for Swift 5 being more strict about some things (#808, #809)
    • Avoid generating code that doesn't compile when enum case aliases have naming collisions in their Swift forms (#822)
    • Drop support for Swift <=3.1 since 4.2 has been the current GM for a while now. (#825)
    Source code(tar.gz)
    Source code(zip)
  • 1.2.0(Oct 17, 2018)

    Changes of interest:

    • #800 Add JSON encoding options: Not, this is a source compatible change, but not binary compatible because of a new parameter with a default, hence the 1.2.0 version numbers.
    • #804 Fix Json map encoding for float, sint, fixed, sfixed
    Source code(tar.gz)
    Source code(zip)
  • 1.1.2(Sep 13, 2018)

    Release to provide full Swift 4.2 toolchain support. Most interesting Pull Requests:

    • Hashable conformance and Hashing changes #784

    The 1.1.0 Release already included #764 (SwiftPM 4.2 specific manifest), #766 (CaseIterable for enums).

    Source code(tar.gz)
    Source code(zip)
  • 1.1.1(Aug 14, 2018)

    • #781 & #783 - avoid extra indirection on equality methods.
    • #785 fixes long standing (and overlooked) issue around clear* methods when the used shared heap storage with another instance.
    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Jul 26, 2018)

    Changes of interest:

    • Support for Swift < 3.1 has been dropped. (#736, #765)
    • Some fixes around text_format parsing (#738)
    • Added missing support for escape sequences in JSON byte data (#744)
    • Start taking advantage of the support from conditional conformances (#747, #749, #755)
    • Fail for unknown JSON fields, but provide an option to ignore them instead (#771)
    • Start Swift 4.2 support (#764, #766). There will be more work for this in future releases.

    Note: This release includes a correction to the handing of unknown fields when parsing JSON messages. Previous releases were not matching the Protobuf Spec which calls for unknown fields to error. The spec does allow for an option to request that unknown fields are silently dropped. So this release includes the fix to error for unknown fields, but adds an option to JSONDecodingOptions to request unknown fields be ignored instead of causing the failure. PR #771 includes these changes and more details.

    Source code(tar.gz)
    Source code(zip)
  • 1.0.3(Mar 2, 2018)

    Release to provide Swift 4.1 toolchain support. Most interesting Pull Requests:

    • #716 Support for Swift 4.1's SE0187 changes.
    • #718 More Swift 4.1 cleanups
    • #719 Swift 4.1 added warnings for "near misses" with default impls of protocols that cause build warning in generated code
    Source code(tar.gz)
    Source code(zip)
Owner
Apple
Apple
Generate protobuf message definitions from Swift structs

SwiftProtobufGen Generates protobuf definitions from Swift structs Building For some reason Xcode builds don't work at all but swiftpm command line bu

null 3 Nov 1, 2021
Swift Server Implementation - RESTful APIs, AWS Lambda Serverless For Swift Runtime amazonlinux: AWS Lambda + API Gateway

Swift Server Implementation - RESTful APIs, AWS Lambda Serverless For Swift Runtime amazonlinux: AWS Lambda + API Gateway deployed on Graviton arm64 build swift:5.6.2-amazonlinux2-docker image

Furqan 2 Aug 16, 2022
💡 A light Swift wrapper around Objective-C Runtime

A light wrapper around Objective-C Runtime. What exactly is lumos? lumos as mentioned is a light wrapper around objective-c runtime functions to allow

Suyash Shekhar 139 Dec 19, 2022
Swift-friendly API for a set of powerful Objective C runtime functions.

ObjectiveKit ObjectiveKit provides a Swift friendly API for a set of powerful Objective C runtime functions. Usage To use ObjectiveKit: Import Objecti

Roy Marmelstein 850 Oct 25, 2022
A simple utility allowing to detect Swift version at runtime.

SwiftVersionDetector SwiftVersionDetector allows you to detect Swift version at runtime. Note that detecting the Swift version of the machine on which

Alessandro Venturini 2 Dec 3, 2022
A Flutter plugin (platform channel with Swift) to get the current app name and also bring our app to the front.

window_to_front A new flutter plugin project. Getting Started This project is a starting point for a Flutter plug-in package, a specialized package th

Minas Giannekas 1 Nov 13, 2021
Swift Package Manager plugin which runs ActionBuilder to create a Github Actions workflow for a swift package.

ActionBuilderPlugin A Swift Package Manager command which builds a Github Actions workflow for the current package. By default the workflow file will

Elegant Chaos 4 Jul 20, 2022
Streamdeck plugin to toggle DND on your Mac

DND Toggler plugin for Stream Deck Plugin for Stream Deck written in Swift (macOS only). Description DND Toggler lets you control macOS DND (Do not di

Francesco Face 11 Oct 31, 2022
A Swift package for rapid development using a collection of micro utility extensions for Standard Library, Foundation, and other native frameworks.

ZamzamKit ZamzamKit is a Swift package for rapid development using a collection of micro utility extensions for Standard Library, Foundation, and othe

Zamzam Inc. 261 Dec 15, 2022
Cross-Platform, Protocol-Oriented Programming base library to complement the Swift Standard Library. (Pure Swift, Supports Linux)

SwiftFoundation Cross-Platform, Protocol-Oriented Programming base library to complement the Swift Standard Library. Goals Provide a cross-platform in

null 620 Oct 11, 2022
Project shows how to unit test asynchronous API calls in Swift using Mocking without using any 3rd party software

UnitTestingNetworkCalls-Swift Project shows how to unit test asynchronous API ca

Gary M 0 May 6, 2022
ResponderChain is a library that passes events using the responder chain.

ResponderChain ResponderChain is a library that passes events using the responder chain.

GodL 21 Aug 11, 2021
TypeStyle is a handy app for iPhone and iPad that generates text using different styles and decorations. It is a native Swift iOS app.

TypeStyle TypeStyle is a handy app for iPhone and iPad that generates text using different styles and decorations. It is a native Swift iOS app. Featu

Eugene Belinski 31 Dec 14, 2022
A simple Pokedex app written in Swift that implements the PokeAPI, using Combine and data driven UI.

SwiftPokedex SwiftPokedex is a simple Pokedex app written by Viktor Gidlöf in Swift that implements the PokeAPI. For full documentation and implementa

Viktor G 26 Dec 14, 2022
This is a app developed in Swift, using Object Oriented Programing, UIKit user interface programmatically, API Request and Kingfisher to load remote images

iOS NOW ⭐ This is a app developed in Swift, using Object Oriented Programing, UIKit user interface programmatically, API Request and Kingfisher to loa

William Tristão de Paula 1 Dec 7, 2021
Parsing indeterminate types with Decodable and Either enum using Swift

Decodable + Either Parsing indeterminate types with Decodable and Either enum us

Alonso Alvarez 1 Jan 9, 2022
WWDCKit - Creating and Using a Swift Package

WWDCKit - Creating and Using a Swift Package 1. Create the Package in Xcode Navigate to File >> New >> Package. Give the Package a name e.g "WWDCKit".

Alex Paul 6 Dec 11, 2022
Monitor changes to files and directories using kernel event notifications (kqueue) in Swift

SKQueue SKQueue is a Swift libary used to monitor changes to the filesystem. It wraps the part of the kernel event notification interface of libc, kqu

Daniel Pedersen 86 Oct 26, 2022
Protected is a Swift Package that allows you to specify the read and write rights for any type, depending on context by using Phantom types

Protected is a Swift Package that allows you to specify the read and write rights for any type, depending on context by using Phantom types

Mathias Quintero 9 Sep 25, 2022