Discussion topics:

• Stream notifications for errors
• Stream close notification
• Cascading notifications throughout streams (up the stream and/or down the stream)
• Multiple future types
• Lightweight versus easy to use futures
• Timing out promises efficiently
• Helpers for managing tasks on the event loop

These topics explained from top to bottom:

Error stream notifications

Currently, stream notifications are handled by registering a closure. You can set a closure on the receiving/draining end of the stream, the result. But anywhere else will break the error chain unless explicitly passed on. This also prevents errors from cascading back. An error in TCP will trigger later in the process of handling the HTTP request, but the other way around doesn't work (Database errors cascading to the HTTP layer). That is still a useful feature, for example, for throwing 500 pages by default.

let requests = socket.stream(to: HTTPParser)
let responses = request.stream(to: myApp)
responses.stream(to: socket)

responses.catch { error in
  print("500 error")
}

// At this moment, the socket does not find any errors, nor do the parser serializer

Stream close notifications

Closing streams is handled the same way, currently. We should decide if we want to keep that this way.

// WebSocket.swift
socket.onClose = {
  websocket.sendCloseMessage()
}

// WebSocket+SSL.swift
socket.onClose = {
  tls.deinitialize()
}

// Chat.swift
socket.onClose = {
  friends.notifyOfflineStatus()
}

Cascading notifications

As discussed in the error stream notification, cascading these errors can interrupt a process causing no additional unnecessary work to be executed. But this must be implemented with care. Interrupting in the middle of a registration form (adding the database entry, sending the email and 2 other ops) can prevent some ops from executing whilst others did execute.

There should be sensible helpers for that.

drop.get {
  return database.all(User.self, in: "SELECT * FROM users")
}

class ErrorMiddleware: Middleware {
  func handle(request: Request, chainingTo responder: Responder) -> Future<Response> {
    let promise = Promise<Response>()

    return responder.respond(to: request).do(promise.complete).catch {
        promise.complete(Response(status: 500))
    }

    return promise.future
  }
}

Multiple future types

As of right now, futures are really easy to use and sensible. They conform to FutureType allowing extensibility and more (generic) integration. One such example is the conformance of Response to FutureType, effectively making Future(Response(...)) unnecessary bloat.

Pseudocode Problem

let users = database.all(User.self, in: "SELECT * FROM users LIMIT 100") // [User].count == 100

If User is a struct of 10KB, [User] will be copied around 2 times with an overhead. Once to the future, once from the future to the handler. This copies around 2MB of extra data.

Timing out promises

Promises can be timed out whilst blocking, currently. But you cannot easily dispatch a task like this without manually interfacing with the DispatchQueue in a worker's eventloop. Having to block for waiting is not sensible, and against the async design. If an HTTP request is waiting for >30 and especially >60 seconds, you're pretty much guaranteed to have timed out. And if any client-side operation done by Vapor (such as calling stripe) times out after X seconds, we shouldn't keep the promise unfulfilled in memory.

Use case

// Throws a timeout after 5 seconds of waiting for a result
return websocket.connect(to: "ws://localhost:8080/").map(timeout: .seconds(5)) { websocket in
  ...
}

Helpers for managing tasks

Finally, playing into this last topic. Many people are using websockets or other features that require cron-jobs. Some people may want to send a notification every minute for new chat messages, regardless of circumstances. Or send a ping to the client every 15 seconds to keep the connection open. They're very common use cases and should be covered from the start, especially with async.

Use case

websocket.worker.every(.seconds(15)) {
  // keep the connection alive
  websocket.ping()
}

Notification Service Extension is an interesting feature that can send plaintext or rich push notifications to iOS.

Is it available on Linux and macOS server side beside the pubsub method?

Steps to reproduce

Create a simple Vapor app that opens WebSocket connections:

import Vapor

extension Application {
  func configure(
    onWebSocketOpen: @escaping (WebSocket) -> (),
    onWebSocketClose: @escaping (WebSocket) -> ()
  ) {
    webSocket("watcher") { _, ws in
      onWebSocketOpen(ws)
      ws.onClose.whenComplete { _ in onWebSocketClose(ws) }
    }
  }
}

final class Server {
  private struct Lifecycle: LifecycleHandler {
    weak var server: Server?

    /// Closes all active WebSocket connections
    func shutdown(_ app: Application) {
      try! EventLoopFuture<()>.andAllSucceed(
        server?.connections.map { $0.close() } ?? [],
        on: app.eventLoopGroup.next()
      ).wait()
      print("shutdown finished")
    }
  }

  private var connections = Set<WebSocket>()
  private let app: Application

  init() throws {
    var env = Environment(name: "development", arguments: ["vapor"])
    try LoggingSystem.bootstrap(from: &env)
    app = Application(env)
    app.configure(
      onWebSocketOpen: { [weak self] in
        self?.connections.insert($0)
      },
      onWebSocketClose: { [weak self] in
        self?.connections.remove($0)
      }
    )
  }

  /// Blocking function that starts the HTTP server
  func run() throws {
    defer { app.shutdown() }
    app.lifecycle.use(Lifecycle(server: self))
    try app.run()
  }
}

try Server().run()

Expected behavior

When this server app is started from command-line and any WebSocket connections are established, the server process can stopped quickly with Ctrl-C.

Actual behavior

The server process can't be stopped quickly with Ctrl-C. It hangs for about 5 seconds or so and then raises the following error:

shutdown finished
[ ERROR ] Could not stop HTTP server: Abort.500: Server stop took too long.
ERROR: Cannot schedule tasks on an EventLoop that has already shut down. This will be upgraded to a forced crash in future SwiftNIO versions.

Environment

These are the details of revelant package dependencies from Package.resolved:

      {
        "package": "vapor",
        "repositoryURL": "https://github.com/vapor/vapor.git",
        "state": {
          "branch": null,
          "revision": "88293674e2ea017691c56af20d0938dfff7ece04",
          "version": "4.27.0"
        }
      },
      {
        "package": "websocket-kit",
        "repositoryURL": "https://github.com/vapor/websocket-kit.git",
        "state": {
          "branch": null,
          "revision": "b0736014be634475dac4c23843811257d86dcdc1",
          "version": "2.1.1"
        }
      }

• Vapor Framework version: 4.27.0, but this was reproducible in versions as old as 4.5.0
• Vapor Toolbox version: not installed
• OS version: macOS 10.15.6

The Vapor 4.0 documentation of "Async" suggest to run any blocking work in the background, such that several clients won't block each other. I've set up a new project from scratch and registered two routes:

app.get("hello-again") { req -> String in
    print("hello-again requested")

    return "Hello, world, again!"
}

app.get("hello") { req -> EventLoopFuture<String> in
    print("hello requested")
    let promise = req.eventLoop.makePromise(of: Void.self)

    req.application.threadPool.submit { _ in
        sleep(5)
        print("(did some work)")

        promise.succeed(Void())
    }

    return promise.futureResult.transform(to: "Hello, world!")
}

If I request GET/hello and immediately GET/hello-again, GET/hello-again is served directly as expected. But if I request GET/hello and than immediately a second GET/hello, the second request waits for the first one to complete. Thus, the output in my console is:

[ INFO ] GET /hello
hello requested
(did some work)
[ INFO ] GET /hello
hello requested
(did some work)

But from the documentation I would have rather expected:

[ INFO ] GET /hello
hello requested
[ INFO ] GET /hello
hello requested
(did some work)
(did some work)

What am I missing?

Another socket issue that occurs after 30 seconds. It happens on Ubuntu 16.04 with Swift 3.0.1 and Vapor 1.1.10 on an EC2 instance running behind an Elastic Load Balancer:

Server error: dispatch(Transport.StreamError.receive("There was a problem while receiving data.", Socket failed with code 11 ("Resource temporarily unavailable") [readFailed] "Unknown error"))

Is it important or can we silence that error, too?

How am I supposed to do this? Vapor requires me to register a type as a controller, not an object. The Controller protocol dictates this type should have a default initializer. Vapor then uses this initializer to create an object right before it calls one of the index, store, ... methods on it. This means I can't inject anything into the object because I can't initialize it.

Wouldn't it be better to register an object as a controller, instead of a type? That's the way Blackfish does it, meaning I can inject my dependencies (model) via the initializer as follows (Blackfish code):

app.use(path: "/api/students", controller: StudentsApiController(model: model))

Steps to reproduce

Tried installing vapor with homebrew with the official docs. Running all commands in zsh:

brew tap vapor/tap
brew install vapor/tap/vapor
vapor --help

Expected behavior

vapor --help running properly without errors

Actual behavior

Environment

• Vapor Framework version:
• Vapor Toolbox version: /usr/local/Cellar/vapor/3.1.10
• OS version: MacOS 10.14.6

Hello, when I was trying to set up a small a vapor project I ran into an issue with the xcode project that gets created. I ran vapor new dragons, cd dragons, vapor xcode. When the xcode project opened up, I added a new file "Battles.swift" to the models folder, and edited the main.swift file to code in the image below. When I tried to build the app via xcode, I got this error "No such module 'Vapor'. " However, I was still able to build and run the vapor project via the command line. Not quite sure what the problem is. It might just be my machine. Any help would be appreciated. Thanks

• Vapor Version: 2.0
• OS: macOS Sierra
• OS Version: 10.12.6
• Vapor Toolbox Version: 2.0.4

the ServerSocket.swift in TLS module the function accept()->Self return self i changed it like this at the end of the function

    let c = InternetSocket(client, context)
    c.cSSL = ssl
    return c

but then server always report error like this

Server error: dispatch(Transport Layer Security Error: Undefined error: 0

Identifier: TLS.TLSError.SSL_read (0))
GET /info
Server error: dispatch(Transport Layer Security Error: The operation did not complete; the same TLS/SSL I/O function should be called again later.

Identifier: TLS.TLSError.SSL_read (-1))

so it can't keep connect alive

I18n support

• Author(s): juanma cabello
• Status: Approved

Introduction

I don't think this feature needs an introduction actually; all of us know what internationalization is and why it's important for Vapor to support it. Anyway, in case this paragraph is mandatory: Vapor needs to support internationalization in order to become the greatest Swift based web framework. 😄

Motivation

One of the key features of every professional web framework is the I18n support. Right now Vapor lacks this feature and I think adding this we can make Vapor to be one step closer to the level of others web frameworks around the web.

Proposed solution

I believe the best place for the dictionaries is under the Resources/Locales folder. The structure to be used to separate each language is quite simple: a folder with the language code (es, en, jp, etc.) and as much files as the developer wants to organise its dictionaries under those folders.

As Vapor has JSON format config files already, I propose to use this file format to store the dictionaries, so we can read them natively.

An I18n class is going to take care about locales and site language switching.

Code snippets

I propose the static func t(_: String) method to do the translation of the key passed as argument.

I18n.t('foo.bar')

As you can see it's quite straightforward. That method just search for the key in the corresponding locale folder.

The static func changeLanguage(_: String) method will change the current selected locale to the language code passed as parameter in the call.

I18n.changeLanguage('en')

Impact

As I18n class is quite atomic I don't expect any impact to Vapor applications.

Alternatives considered

I think about using yaml instead json due to the vast use of yaml format in web frameworks but json is natively supported by Swift so Vapor get less dependencies to third party libraries this way.

Decision (For Moderator Use)

On May 26th, 2016, the community decided to approve this proposal.

Currently every project created with vapor new is based on the vapor example project. That results in boilerplate code which is more suitable for demonstration purposes than actual starting ground for new projects. In example the mustache example functionality and dependency likely is going to be removed every time.

The solution would be to introduce the example argument for the vapor command line application, which creates an new project based on the example. In parallel the new argument will be modified to create a barebone project with just the minimal content necessary to get going (requires a new repository as a source). In addition there is an option (argument --template, other suggestions welcome) to specify the repository to use as the template for creating a new project.

This enables new starting points when creating new projects.

Example

vapor new Vaporizer

Creates a new project in the Vaporizer directory based on the minimal, default project template. Invokes dependency installation and build automatically (vapor build).

vapor new Vaporizer --template=https://github.com/alomvar/vapor-blog-template

Creates a new project in the Vaporizer directory based on the provided project template (a Git repository). Invokes dependency installation and build automatically (vapor build).

vapor example

Creates a new project in VaporExample based on the official Vapor example project. Invokes dependency installation and build automatically (vapor build) and runs it afterwards.

Describe the bug

The WebSocket route’s shouldUpgrade closure can throw an error. The closure can also return nil, which would cause NIOWebSocketServerUpgrader to throw NIOWebSocketUpgradeError.unsupportedWebSocketTarget.

The error is logged by the catch-all HTTPServerErrorHandler channel handler. However, there is no response sent to the client.

To Reproduce

routes.webSocket { request in
   throw Abort(.badRequest)
} onUpgrade: { (request, webSocket) in
}

or

routes.webSocket { request in
   return nil
} onUpgrade: { (request, webSocket) in
}

Expected behavior

An appropriate HTTP response should be returned to the client. Perhaps we could even reuse ErrorMiddleware to generate the response.

Environment

• Vapor Framework version: 4.67.5
• Vapor Toolbox version: 18.6.0
• OS version: macOS 12.6.2

Describe the bug

When i try to upload a file . Server Crash . Please see attached screenshot

This issue may be related to https://github.com/vapor/multipart-kit/issues/86

To Reproduce

Steps to reproduce the behavior:

1. Upload a file larger than 1 mb

Environment

• Vapor Framework version: 4.67.0
• Vapor Toolbox version:
• OS version: Mac Mini M1 (Ventura 13.0)

Additional context

Add any other context about the problem here.

Describe the bug

Vapor has the option for chunk streaming of the body.

Response(body: .init(stream: { writer in
})

People may want to use async functions to get the results which they will write in the response Because this function doesn't accept async code, it needs to be wrapped inside Task. But for writing buffer Vapor uses writeAndFlush function from NIO. This function calls eventLoop.assertInEventLoop and it asserts self.inEventLoop This returns true if the current NIOThread is the same as the NIOThread that is tied to this EventLoop. Although code builds successfully, at runtime it crashes through Task thread and event loop thread are different

I have created a spy client, in order to stub the response and also being able to verify the request.

To reduce boilerplate code, I created a XCTVaporTestCase. You don't need to configure the application every time. And tearing it down. All is being handled by XCTVaporTestCase.

You just simply do this:

final class ExampleTests: XCTVaporTestCase {
     func test_example() throws {
           try client.stubResponse(httpStatus: .ok)
     }
}

Bug Description

I'm probably holding it wrong: It seems a server can not configure a new WebSocket before Vapor establishes the WebSocket connection. This means the client requesting the connection receives it before the server even had a chance to configure the WebSocket, i.e. before the WebSocket even has its input handler set.

This has led to a flaky and hard to find (infuriating) bug, where the client's first sent data sometimes got totally lost because it arrived 15 milliseconds too early. My preliminary workaround is to let the client wait 100 ms to increase the probability that the server has its stuff together when the client actually uses the WebSocket connection.

To Reproduce

Establish a WebSocket connection between client and server and let the client send data immediately after its successful WebSocket request.

Server side:

myRouteBuilder.webSocket("websocket") { request in
    HTTPHeaders()
} onUpgrade: { request, newWebsocket in
    // 🐢 Too late. Vapor has already established the connection, so the client has already sent data.
    newWebsocket.onBinary { ws, bufferedBytesFromWebSocket in
        // do somethin with the incoming data
    }
}

The workaround in the client, where it waits before sending the first data:

let server = try await createServer(forLanguageNamed: codebase.languageName)
try await Task.sleep(nanoseconds: 100_000_000)
_ = try await server.request(.initialize(folder: codebase.folder))

Expected behavior

I expect to be able to configure the WebSocket before it is offered to the client. Either onUpgrade would be called before the connection is finally established with the client, or there would be some other means to get a hold of the WebSocket – ideally even before deciding on whether to upgrade the connection, because that decision can depend on whether the WebSocket can even be configured. I honestly (naively) would expect that in shouldUpgrade I can create and return the WebSocket that is then passed to onUpgrade.

Also, I would expect the documentation of Vapor WebSockets to reflect the current implementation – for example changes made in April 2021 and in October 2020.

Environment

• macOS 13.1
• Xcode 14.2
• Vapor 4.67.5
• The client uses URLSessionWebSocketTask

Invalid HTTP previously just closed the connection, but did not actually handle the errors.

This PR uses an adapted version of https://github.com/apple/swift-nio/blob/main/Sources/NIOHTTP1/HTTPServerProtocolErrorHandler.swift to properly catch exceptions if they occur.

Closes: https://github.com/vapor/vapor/issues/2921