Network testing à la VCR in Swift

Overview

Vinyl

Version Build Status codecov.io Carthage Swift 4.2 License MIT platforms

Vinyl is a simple, yet flexible library used for replaying HTTP requests while unit testing. It takes heavy inspiration from DVR and VCR.

Vinyl should be used when you design your app's architecture with Dependency Injection in mind. For other cases, where your URLSession is fixed, we would recommend OHHTTPStubs or Mockingjay.

How to use it

Carthage

github "Velhotes/Vinyl"

Intro

Vinyl uses the same nomenclature that you would see in real life, when playing a vinyl:

  • Turntable
  • Vinyl
  • Track

Let's start with the most basic configuration, where you already have a track (stored in the vinyl_single):

let turntable = Turntable(vinylName: "vinyl_single")
let request = URLRequest(url: URL(string: "http://api.test.com")!)

turntable.dataTask(with: request) { (data, response, anError) in
 // Assert your expectations
}.resume()

A track is a mapping between a request (URLRequest) and a response (HTTPURLResponse + Data? + Error?). As expected, the vinyl_single that you are seeing in the example above is exactly that:

[
  {
    "request": {
        "url": "http://api.test.com"
    },
    "response": {
        "url": "http://api.test.com",
        "body": "hello",
        "status": 200,
        "headers": {}
    }
  }
]

Vinyl by default will use the mapping approach. Internally, we will try to match the request sent with the track recorded based on:

  • The sent request's url with the track request's url.
  • The sent request's httpMethod with the track request's httpMethod.

As you might have noticed, we don't provide an httpMethod in the vinyl_single, by default it will fallback to GET.

If the mapping doesn't suit your needs, you can customize it by:

enum RequestMatcherType {
    case method
    case url
    case path
    case query
    case headers
    case body
    case custom(RequestMatcher)
}

In practise it would look like this:

let matching = MatchingStrategy.requestAttributes(types: [.body, .query], playTracksUniquely: true)
let configuration = TurntableConfiguration(matchingStrategy:  matching)
let turntable = Turntable(vinylName: "vinyl_simple", turntableConfiguration: configuration)

In this case we are matching by .body and .query. We also provide a way of making sure each track is only played once (or not), by setting the playTracksUniquely accordingly.

If the mapping approach is not desirable, you can make it behave like a queue: the first request will match the first response in the array and so on:

let matching = MatchingStrategy.trackOrder
let configuration = TurntableConfiguration(matchingStrategy:  matching)
let turntable = Turntable(vinylName: "vinyl_simple", turntableConfiguration: configuration)

We also allow creating a track by hand, instead of relying on a JSON file:

let track = TrackFactory.createValidTrack(url: URL(string: "http://feelGoodINC.com")!, body: data, headers: headers)

let vinyl = Vinyl(tracks: [track])
let turntable = Turntable(vinyl: vinyl, turntableConfiguration: configuration)

If you have a custom configuration that you would like to see shared among your tests, we recommend the following:

class FooTests: XCTestCase {
    let turntable = Turntable(turntableConfiguration: TurntableConfiguration(matchingStrategy: .trackOrder))

    func test_1() {
       turntable.loadVinyl("vinyl_1")
       // Use the turntable
    }

    func test_2() {
       turntable.loadVinyl("vinyl_1")
       // Use the turntable
    }
}

This approach cuts the unnecessary boilerplate (you will also feel like a 🎶 Dj 🎶 )

Coming from Alamofire

Instead of using the default manager, initialize a new one via:

public init?(
    session: URLSession,
    delegate: SessionDelegate,
    serverTrustPolicyManager: ServerTrustPolicyManager? = nil)
    {
        guard delegate === session.delegate else { return nil }

        self.delegate = delegate
        self.session = session

        commonInit(serverTrustPolicyManager: serverTrustPolicyManager)
    }

Your network layer, could then be in the form of:

class Network {
    private let manager: SessionManager

    init(session: URLSession) {
        self.manager = SessionManager(session: session, delegate: SessionDelegate())
    }
}

This way it's becomes quite easy to test your components using Vinyl. This might be too cumbersome for some users, so don't forget that you still have the URLProtocol approach (with OHHTTPStubs and Mockingjay).

Coming from DVR

If your tests are already working with DVR, you will probably have pre-recorded cassettes. Vinyl provides a compatibility mode that allows you to re-use those cassettes.

If your tests look like this:

let session = Session(cassetteName: "dvr_single")

You can just use a Turntable instead:

let turntable = Turntable(cassetteName: "dvr_single")

That way you won't have to throw anything away.

Note: only use it for cassettes that you already have in the bundle, otherwise recording will crash trying to read missing file.

Recording

You can also use Vinyl to record requests and responses from the network to use for future testing. This is an easy way to create Vinyls and Tracks automatically with genuine data rather than creating them manually.

There are 3 recording modes:

  • .none - recording is disabled.
  • .missingVinyl - will record a new Vinyl if the named Vinyl does not exist. This is the default mode.
  • .missingTracks - will record new Tracks to an existing Vinyl where the Track is not found.

Both .missingVinyl and .missingTracks allow you to specify a recordingPath for where to save the recordings (this should be a file path). If the path is not provided (nil) then the default path is current test target's Resource Bundle, which is also the default location from which Vinyl's are loaded.

A simple example

let recordingMode = RecordingMode.missingVinyl(recordingPath: nil)
let configuration = TurntableConfiguration(recordingMode: recordingMode)
let turntable = Turntable(vinylName: "new_vinyl", turntableConfiguration: configuration)
let request = URLRequest(url: URL(string: "http://api.test.com")!)

turntable.dataTask(with: request) { (data, response, anError) in
    // Assert your expectations
}.resume()

The recordingMode in the example above is actually the default, but it's shown explicitly to make it clearer. With the above configuration, if "new_vinyl.json" does't exist it is created and the request will be made over the network. Both the request and response will be recorded.

Recordings are saved either when the Turntable is deinitialized or you can explicitly call turntable.stopRecording() which will persist the recorded data.

You can provide a URLSession for a Turntable to use for making network requests:

let turntable = Turntable(vinylName: "new_vinyl", turntableConfiguration: configuration, urlSession: aSession)

If no URLSession is provided, it defaults to URLSession.shared.

Current Status

The current version is currently being used in a project successfully. This gives us some degree of confidence it will work for you as well. Nevertheless don't forget this is a pre-release version. If there is something that isn't working for you, or you are finding its usage cumbersome, please let us know.

Roadmap

  • Allow the user to configure how strict the library should be.
  • Allow the user to define their own response without relying on a json file.
  • Instead of mapping requests ➡️ responses , fix the responses in an array (e.g. first request made will use the first response in the array and so on).
  • Allow request recording. (#12)
  • Debug mode (#28)

Why not simply use DVR?

From our point of view, DVR is too strict. If you change something in your request, even if you are expecting the same response, your tests will break. With that in mind, we intend to follow VCR's approach, where you can define what should be fixed, and what's not (e.g. only care if the NSURL changes, instead of the headers, body and HTTP Method). Bottom line, our approach will have flexibility and extensibility in mind.

We also feel that the DVR project has stalled. As of 15/02/2016, the project has 10 issues open, 2 PRs and the last commit was more than one month ago.

Contributing

We will gladly accept Pull Requests that take the roadmap into consideration. Documentation, or tests, are always welcome as well. ❤️

Comments
  • Allow a Turntable to load a Vinyl on demand

    Allow a Turntable to load a Vinyl on demand

    Looking at this usage from a usage POV, I seem myself creating the Turntable with a given configuration and then on each test case just load a particular vinyl and use it. As it is right now, you have to create a turntable every time.

    Improvement Proposal 
    opened by RuiAAPeres 19
  • Alamofire Support

    Alamofire Support

    tl;dr Alamofire makes extensive use of the NSURLSessionDelegate protocol. Vinyl did not support this.

    What's changing

    • [x] Expand URLSessionDataTask to better implement the properties and methods described in the base class NSURLSessionTask
    • [x] Trigger the appropriate delegate calls in URLSessionDataTask as described by the documentation in NSURLSessionDataTask
    • [x] Accept an Optional<NSURLSessionDelegate> as an argument to Turntable initializers
    • [x] Expand the test suite to cover the new delegate pattern
    • [x] Rebase and squash the commits to make it a little bit more coherent

    Example Usage

    class CredentialTests: XCTestCase {
    
        var turntable: Turntable!
    
        var manager: Alamofire.Manager!
    
        override func setUp() {
            super.setUp()
            // Put setup code here. This method is called before the invocation of each test method in the class.
    
            let types: [RequestMatcherType] = [.Body, .Query]
            let matching: MatchingStrategy = .RequestAttributes(types: types, playTracksUniquely: true)
            let config = TurntableConfiguration(matchingStrategy:  matching)
            let delegate = Alamofire.Manager.SessionDelegate()
            self.turntable = Turntable(vinylName: "credentials", turntableConfiguration: config, delegate: delegate)
            self.manager = Alamofire.Manager(session: turntable, delegate: delegate)
    
            self.continueAfterFailure = false
            XCTAssertNotNil(turntable)
            XCTAssertNotNil(manager)
            self.continueAfterFailure = true
        }
    
        func testFailureToProvidePostBody() {
            let expectation = self.expectationWithDescription(#function)
    
            manager?
                .request(.POST, "https://json.schedulesdirect.org/20141201/token")
                .responseString { response in
                    XCTAssertEqual(response.result.value, "{\"response\":\"Username expected but not provided.\",\"code\":403}")
                    XCTAssertEqual(response.response?.statusCode, 403)
                    expectation.fulfill()
            }
    
            self.waitForExpectationsWithTimeout(5.0) { (error) in
                if let error = error {
                    XCTFail("Expectation failed with error: \(error)")
                }
            }
        }
    
    }
    
    Work in Progress Improvement 
    opened by RLovelett 12
  • caught

    caught "NSInvalidArgumentException"

    I'm trying to integrate Alamofire and Vinyl.

    In my test case I'm doing:

    let turntable = Turntable(vinylName: "vinyl_single", delegateQueue: NSOperationQueue.mainQueue())
    print(turntable.delegate)
    

    Which crashes on the print statement with the following exception.

    /Users/lovelett/Source/lineup/LineupTests/LineupTests.swift:34: error: -[LineupTests.LineupTests testVinyl] : failed: caught "NSInvalidArgumentException", "-[Vinyl.Turntable delegate]: unrecognized selector sent to instance 0x7fa8c3017380"
    (
        0   CoreFoundation                      0x00000001105f80e5 __exceptionPreprocess + 165
        1   libobjc.A.dylib                     0x000000011006ddeb objc_exception_throw + 48
        2   CoreFoundation                      0x000000011060109d -[NSObject(NSObject) doesNotRecognizeSelector:] + 205
        3   CoreFoundation                      0x0000000110547e1a ___forwarding___ + 970
        4   CoreFoundation                      0x00000001105479c8 _CF_forwarding_prep_0 + 120
        5   LineupTests                         0x000000011a4dcd0d _TFC11LineupTests11LineupTests9testVinylfT_T_ + 269
        6   LineupTests                         0x000000011a4dce32 _TToFC11LineupTests11LineupTests9testVinylfT_T_ + 34
        7   CoreFoundation                      0x00000001104e16ec __invoking___ + 140
        8   CoreFoundation                      0x00000001104e153e -[NSInvocation invoke] + 286
        9   XCTest                              0x000000010fb28eab __24-[XCTestCase invokeTest]_block_invoke_2 + 362
        10  XCTest                              0x000000010fb5c34d -[XCTestContext performInScope:] + 190
        11  XCTest                              0x000000010fb28d30 -[XCTestCase invokeTest] + 169
        12  XCTest                              0x000000010fb2935a -[XCTestCase performTest:] + 459
        13  XCTest                              0x000000010fb2636f -[XCTestSuite performTest:] + 396
        14  XCTest                              0x000000010fb2636f -[XCTestSuite performTest:] + 396
        15  XCTest                              0x000000010fb13180 __25-[XCTestDriver _runSuite]_block_invoke + 51
        16  XCTest                              0x000000010fb34ad8 -[XCTestObservationCenter _observeTestExecutionForBlock:] + 640
        17  XCTest                              0x000000010fb130c5 -[XCTestDriver _runSuite] + 453
        18  XCTest                              0x000000010fb13e41 -[XCTestDriver _checkForTestManager] + 259
        19  XCTest                              0x000000010fb5d6b8 _XCTestMain + 628
        20  xctest                              0x000000010fa9d613 xctest + 5651
        21  libdyld.dylib                       0x0000000112ed59e9 start + 1
        22  ???                                 0x0000000000000005 0x0 + 5
    )
    

    In case it helps with the trouble-shooting. The entire source can be found here on the vinyl branch.

    Suggestions on what I am doing wrong?

    Bug 
    opened by RLovelett 7
  • AlamoFire example wrong?

    AlamoFire example wrong?

    You mention a setup example for working with AlamoFire here but this seems to look over the fact that the recommended init on Manager validates that delegate === session.delegate.

    Am I missing something here or this doesn't really work with AlamoFire?

    Question 
    opened by crayment 7
  • Swift 3.1 warnings

    Swift 3.1 warnings

    Vinyl/Response.swift:63:41: String interpolation produces a debug description for an optional value; did you mean to make this explicit?

    Vinyl/Response.swift:64:43: String interpolation produces a debug description for an optional value; did you mean to make this explicit?

    Vinyl/Response.swift:66:18: String interpolation produces a debug description for an optional value; did you mean to make this explicit?

    opened by brentleyjones 4
  • Add configuration override

    Add configuration override

    Without this override accessing configuration will crash with -[Vinyl.Turntable _local_immutable_configuration]: unrecognized selector sent to instance.

    opened by brentleyjones 4
  • Migrated to Swift 3

    Migrated to Swift 3

    Everything has been migrated to Swift 3. Tests are passing.

    SwiftCheck was removed as a git submodule and made a regular Carthage dependency and added as a linked framework for the test targets.

    No function names were changed to match the new Swift naming style, but this could be added to the PR.

    opened by mluisbrown 4
  • Use SwiftCheck for Testing

    Use SwiftCheck for Testing

    I took a crack at #14 to see what would happen. Naturally, this is just a start and there's certainly some rough edges to work out, but it think it turned out quite well.

    Work in Progress Improvement 
    opened by CodaFi 4
  • Use an operation queue to execute completion blocks

    Use an operation queue to execute completion blocks

    This is to replicate the actual behaviour of NSURLSession, as client code might assume this (for example using dispatch_sync on the main queue and dead-locking)

    opened by Sephiroth87 3
  • Generate releases on Travis

    Generate releases on Travis

    We could use Travis to manage our releases, every tag created automatically triggers a build on Travis which can then build and deploy our frameworks to GitHub.

    Reference: GitHub Releases Uploading.

    Infrastructure 
    opened by dmcrodrigues 3
  • Global Turntable Configuration

    Global Turntable Configuration

    @dmcrodrigues @SandroMachado let me know if this feels right. In a test suit, I would image that you would just set up once the configuration Globally for example on a setUp method. But on the other this doesn't feel completly right... :-1:

    What I think it would make more sense is the previous idea:

    class Tests: XCTestCase {
    
       let turntable: Turntable =  // Turntable with a given config
    
       func test_() {
          turntable.loadVinyl(...) // vinyl name
          turntable.loadCassette(...) // cassette name
          turntable.loadVinyl(...) // array of tracks
       }
    }
    
    opened by RuiAAPeres 3
  • Application API only and Apple Watch

    Application API only and Apple Watch

    Amazing library,

    Making framework uses only app extension api only removes a warning generated by Xcode when integrating the library using carthage to a framework.

    Also added support to apple watch with minimum version of 6.0.

    Does this makes sense? Thank you.

    opened by dchohfi 0
  • Allow recording in replay mode when no recording found

    Allow recording in replay mode when no recording found

    Replay configuration can allow to record requests when recordings are missing instead of raising exception. This will simplify the workflow as will require only to remove outdated recordings from the bundle without changing tests code to switch from replay mode to record mode.

    opened by ilyapuchka 0
  • Silently fails to save recording if `recordingPath` contains nonexistent parent directories

    Silently fails to save recording if `recordingPath` contains nonexistent parent directories

    Hi,

    Thanks for your effort, the library is really easy to use and is a great help!

    Currently,

    let recordingPath = "\(nonexistentParentDirectory)/\(testName).json"
    let recordingMode = RecordingMode.missingVinyl(recordingPath: recordingPath)
    

    will result in Vinyl's Recorder silently failing to save newly recorded vinyl.

    Suggest adding an error case into TurntableError for throwing there instead of silent return.

    Happy to send a PR.

    opened by Kastet 1
  • Crash when accessing URLSessionDataTask

    Crash when accessing URLSessionDataTask

    I'm currently testing purposely a failure on a request and when accessing the request on the URLSessionDataTask, it results in a unrecognized selector sent.

    Step to reproduce:

    • Do a request that will expect to fail
    • Try to access task.currentRequest
    • Your test suite will crash with -[Vinyl.URLSessionDataTask currentRequest]: unrecognized selector sent to instance 0x608000058f90

    I haven't looked deeply in the problem, but my first thought was that it is really odd as Vinyl.URLSessionDataTask is a subclass of Foundation.URLSessionDataTask and so it shouldn't be a problem to access currentRequest. Any thought on that?

    opened by Nonouf 3
  • CocoaPods support

    CocoaPods support

    The podspec is already available in the repository and it can be used like this:

    pod 'Vinyl', :git => '[email protected]:Velhotes/Vinyl.git', :branch => 'master'
    

    We're planning to publish in version 1.0.

    Infrastructure 
    opened by dmcrodrigues 0
Releases(0.10.4)
Owner
Tools for Gentlemen
null
SwiftCheck is a testing library that automatically generates random data for testing of program properties

SwiftCheck QuickCheck for Swift. For those already familiar with the Haskell library, check out the source. For everybody else, see the Tutorial Playg

TypeLift 1.4k Dec 21, 2022
Switchboard - easy and super light weight A/B testing for your mobile iPhone or android app. This mobile A/B testing framework allows you with minimal servers to run large amounts of mobile users.

Switchboard - easy A/B testing for your mobile app What it does Switchboard is a simple way to remote control your mobile application even after you'v

Keepsafe 287 Nov 19, 2022
Network testing for Swift

DVR DVR is a simple Swift framework for making fake NSURLSession requests for iOS, watchOS, and OS X based on VCR. Easy dependency injection is the ma

Venmo 650 Nov 3, 2022
Stub your network requests easily! Test your apps with fake network data and custom response time, response code and headers!

OHHTTPStubs OHHTTPStubs is a library designed to stub your network requests very easily. It can help you: test your apps with fake network data (stubb

Olivier Halligon 4.9k Dec 29, 2022
The Swift (and Objective-C) testing framework.

Quick is a behavior-driven development framework for Swift and Objective-C. Inspired by RSpec, Specta, and Ginkgo. // Swift import Quick import Nimbl

Quick 9.6k Dec 31, 2022
Implementing and testing In-App Purchases with StoreKit2 in Xcode 13, Swift 5.5 and iOS 15.

StoreHelper Demo Implementing and testing In-App Purchases with StoreKit2 in Xcode 13, Swift 5.5, iOS 15. See also In-App Purchases with Xcode 12 and

Russell Archer 192 Dec 17, 2022
📸 Delightful Swift snapshot testing.

?? SnapshotTesting Delightful Swift snapshot testing. Usage Once installed, no additional configuration is required. You can import the SnapshotTestin

Point-Free 3k Jan 3, 2023
Snapshot testing tool for iOS and tvOS

SnapshotTest is a simple view testing tool written completely in Swift to aid with development for Apple platforms. It's like unit testing for views.

Pär Strindevall 44 Sep 29, 2022
UI Testing Cheat Sheet and Examples.

UI Testing Cheat Sheet This repository is complementary code for my post, UI Testing Cheat Sheet and Examples. The post goes into more detail with exa

Joe Masilotti 2.1k Dec 25, 2022
Mockingbird was designed to simplify software testing, by easily mocking any system using HTTP/HTTPS

Mockingbird Mockingbird was designed to simplify software testing, by easily mocking any system using HTTP/HTTPS, allowing a team to test and develop

FARFETCH 183 Dec 24, 2022
Automatic testing of your Pull Requests on GitHub and BitBucket using Xcode Server. Keep your team productive and safe. Get up and running in minutes. @buildasaur

Buildasaur Automatic testing of your Pull Requests on GitHub and BitBucket using Xcode Server. Keep your team productive and safe. Get up and running

Buildasaurs 774 Dec 11, 2022
Fastbot is a model-based testing tool for modeling GUI transitions to discover app stability problems

Fastbot is a model-based testing tool for modeling GUI transitions to discover app stability problems. It combines machine learning and reinforcement learning techniques to assist discovery in a more intelligent way.

Bytedance Inc. 446 Dec 29, 2022
Genything is a framework for random testing of a program properties.

Genything is a framework for random testing of a program properties. It provides way to random data based on simple and complex types.

Just Eat Takeaway.com 25 Jun 13, 2022
For Testing APIs of NYTimes

NYTimes-APIs For Testing APIs of NYTimes Mark Dennis Diwa ?? To run the app: Open terminal first then run pod install. Open workspace. Run the app on

Mark Dennis Diwa 0 Nov 23, 2021
A Mac and iOS Playgrounds Unit Testing library based on Nimble.

Spry Spry is a Swift Playgrounds Unit Testing library based on Nimble. The best thing about Spry is that the API matches Nimble perfectly. Which means

Quick 327 Jul 24, 2022
Multivariate & A/B Testing for iOS and Mac

This library is no longer being maintained. You can continue to use SkyLab in your projects, but we recommend switching another solution whenever you

Mattt 792 Dec 15, 2022
Remote configuration and A/B Testing framework for iOS

MSActiveConfig v1.0.1 Remote configuration and A/B Testing framework for iOS. Documentation available online. MSActiveConfig at a glance One of the bi

Elevate 78 Jan 13, 2021
AB testing framework for iOS

ABKit Split Testing for Swift. ABKit is a library for implementing a simple Split Test that: Doesn't require an HTTP client written in Pure Swift Inst

Recruit Marketing Partners Co.,Ltd 113 Nov 11, 2022
AppiumLibrary is an appium testing library for RobotFramework

Appium library for RobotFramework Introduction AppiumLibrary is an appium testing library for Robot Framework. Library can be downloaded from PyPI. It

Serhat Bolsu 327 Dec 25, 2022