BDD Framework and test runner for Swift projects and playgrounds

Last update: Jun 6, 2022

Spectre

Build Status

Special Executive for Command-line Test Running and Execution.

A behavior-driven development (BDD) framework and test runner for Swift projects and playgrounds. It's compatible with both OS X and Linux.

Usage

describe("a person") {
  let person = Person(name: "Kyle")

  $0.it("has a name") {
    try expect(person.name) == "Kyle"
  }

  $0.it("returns the name as description") {
    try expect(person.description) == "Kyle"
  }
}

Reporters

Spectre currently has two built-in reporters, Standard and the Dot reporter. Custom reporters are supported, make a type that conforms to Reporter.

The default reporter can be configured via an environment variable. For example:

$ env SPECTRE_REPORTER=dot swift test
$ env SPECTRE_REPORTER=tap swift test

Standard

The standard reporter produces output as follows:

Passing Tests

Standard Reporter Success

Failing Tests

Standard Reporter Failure

Dot

Using the -t argument, you can use the dot reporter.

Passing Tests

Dot Reporter Success

Failing Tests

Dot Reporter Failure

Expectation

Equivalence

try expect(name) == "Kyle"
try expect(name) != "Kyle"

Truthiness

try expect(alive).to.beTrue()
try expect(alive).to.beFalse()
try expect(alive).to.beNil()

Error handling

try expect(try write()).toThrow()
try expect(try write()).toThrow(FileError.NoPermission)

Comparable

try expect(5) > 2
try expect(5) >= 2
try expect(5) < 10
try expect(5) <= 10

Types

try expect("kyle").to.beOfType(String.self)

Causing a failure

throw failure("Everything is broken.")

Custom assertions

You can easily provide your own assertions, you just need to throw a failure when the assertion does not meet expectaions.

Examples

The following projects use Spectre:

Installation / Running

Swift Package Manager

Check out Commander as an example.

Playground

You can use Spectre in an Xcode Playground, open Spectre.playground in this repository, failures are printed in the console.

Spectre in an Xcode Playground

GitHub

https://github.com/kylef/Spectre
Comments
  • 1. Xcode reports

    Resolves #2

    This PR adds a very naive support for reporting test failures in Xcode. Instead of using runtime to create tests, as suggested in #17 I've added a method XCTestCase.describe that will store a reference to this test case in custom reporter, to use for reporting later, calls global describe function with passed in closure and then runs all the tests. At the end all added tests are removed so that they are not run again later. All individual test failures will appear in Issues navigator.

    Tests will be still run at the application exit (without reporting via Xcode) but if they were already run with XCTestCase.describe they will be removed from the global context and so will not be run twice.

    Also added option to disable colours in output which make output less readable in Xcode console, that does not support ANSI colors anyway.

    screen shot 2017-12-25 at 17 45 57

    Reviewed by ilyapuchka at 2017-12-25 16:47
  • 2. Xcode12 support

    XCTestCase.recordFailure(..) method has been deprecated on Xcode12+ and it will cause compile error on Xcode12.

    スクリーンショット 2020-07-05 0 41 33

    To avoid that, added @available block. In the future, XCTestCase. recordFailure(..) should be replaced with new API, XCTestCase.record(_:).

    Video https://developer.apple.com/videos/play/wwdc2020/10687/ Document https://developer.apple.com/documentation/xctest/xctestcase/3546549-record

    Reviewed by kenmaz at 2020-07-04 15:40
  • 3. Invalid Exclude '…/SourcePackages/checkouts/Spectre/Sources/Spectre/[email protected]': File not found.

    Some dependency of mine has Spectre as a dependency, and when Xcode 13 beta resolves the dependencies, it issues this warning:

    Invalid Exclude '…/SourcePackages/checkouts/Spectre/Sources/Spectre/[email protected]': File not found.
    

    I'm not sure what it means, but this seemed like a reasonable place to report the issue.

    Reviewed by JetForMe at 2021-06-16 23:32
  • 4. Build XCTest extensions on linux

    Nowadays, swift-corelibs-xctest exists, so it should be possible to use Spectre from XCTest on Linux. This change removes the #if guards around XCTest extensions, with the exception of some Xcode 12+ APIs that have not yet made it upstream.

    Testing done

    Builds and tests on a linux machine using the official Swift docker images:

    $ docker pull swift
    $ docker run -itv $PWD:/code swift swift test --package-path /code
    
    Reviewed by elliottwilliams at 2020-11-07 01:27
  • 5. Bump the 0.8.0 to avoid problems with missed files in the playground

    Playground sources a still not available when using 0.8.0 as a dependency. More - various samples (like https://github.com/IBM/FoodTrackerBackend) cannot be correctly configured when using the Swift PM in the CLion (we are currently using CMake for integrating it and CMake complains about files missed). Please, bump the version so the most recent sources are included.

    Reviewed by yeswolf at 2018-05-25 11:15
  • 6. More expectations

    • beNotNil to validate that value is not nil
    • notThrow to validate that function does not throw
    • throw(_ match: (Error) -> Bool) to validate that function throws error using custom condition rather than Equatable
    • throw and throw(_:) expectations to better match other APIs (where to is omitted)
    • expect(...).to.not for not expectations, expect(...).to has now better scoped methods and code completion
    Reviewed by ilyapuchka at 2017-12-25 17:46
  • 7. Fix build errors when compiling with Xcode 9 GM

    When building with Xcode 9 GM, we get the following errors:

    [[email protected] Spectre (ehyche_fix_swift4)]$ swift build
    Compile Swift Module 'Spectre' (8 sources)
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:98:22: warning: redundant conformance constraint 'Key': 'Equatable'
    public func == <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
                         ^
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:98:57: note: conformance constraint 'Key': 'Equatable' inferred from type here
    public func == <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
                                                            ^
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:108:22: warning: redundant conformance constraint 'Key': 'Equatable'
    public func != <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
                         ^
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:108:57: note: conformance constraint 'Key': 'Equatable' inferred from type here
    public func != <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
                                                            ^
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:98:22: warning: redundant conformance constraint 'Key': 'Equatable'
    public func == <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
                         ^
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:98:57: note: conformance constraint 'Key': 'Equatable' inferred from type here
    public func == <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
    
    

    These changes fix those build errors. I also verified that it still builds against Swift 3.2 in Xcode 8.

    Reviewed by ehyche at 2017-09-14 18:43
  • 8. Updates for future versions of Swift

    Few last Swift development snapshots are yapping about this:

    ./Spectre/Sources/Expectation.swift:70:25: warning: string interpolation produces a debug description for an optional value; did you mean to make this explicit?
        throw lhs.failure("\(value) is equal to \(rhs)")
                            ^~~~~~~
    ./Spectre/Sources/Expectation.swift:70:26: note: use 'String(describing:)' to silence this warning
        throw lhs.failure("\(value) is equal to \(rhs)")
                            ~^~~~~~
                             String(describing:  )
    ./Spectre/Sources/Expectation.swift:70:26: note: provide a default value to avoid this warning
        throw lhs.failure("\(value) is equal to \(rhs)")
                            ~^~~~~~
                                   ?? <#default value#>
    

    This PR makes Swift a good tail-waggling boy... :wink: All builds and tests passed.

    Swift 3.0
    [x] - Build
    [x] - Test
    Swift 3.0.1
    [x] - Build
    [x] - Test
    Swift DEV SNAPSHOT 2016-11-11
    [x] - Build
    [x] - Test
    

    @kylef Please review and merge. If you do, please bump the Spectre version tag as well. Thanks.

    Reviewed by rbukovansky at 2016-11-14 09:28
  • 9. Ability to start tests from Xcode

    All of Spectre tests added to global contexts start in atexit. XCode always says that they are passed. As really Spectre tests has not started yet.

    If we start them manually with run. XCode also fails, as inside run exit is called.

    Is there any plan to add ability testing with XCode?

    Reviewed by Igor-Palaguta at 2016-10-20 08:15
  • 10. Support type comparison

    First of all, thank you for working on this, very neat tool.

    I just tried to compare the object type with my expectation. I got it to work with

    https://github.com/ryuichis/swift-ast/blob/66bb167818d4ecc5b009f4f19085826e861d37f5/Tests/declaration/ParsingTypeAliasDeclarationSpec.swift#L119

    However, if I change the line to

    try expect(Mirror(reflecting: node.type).subjectType) == testType

    then I got the following error:

    Tests/declaration/ParsingTypeAliasDeclarationSpec.swift:119:71: error: binary operator '==' cannot be applied to operands of type 'Expectation<Any.Type>' (aka 'Expectation<protocol<>.Type>') and 'Type.Type'
                    try expect(Mirror(reflecting: node.type).subjectType) == testType
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~
    Tests/declaration/ParsingTypeAliasDeclarationSpec.swift:119:71: note: overloads for '==' exist with these partially matching parameter lists: (FloatingPointClassification, FloatingPointClassification), (_MirrorDisposition, _MirrorDisposition), (Mirror.DisplayStyle, Mirror.DisplayStyle), (Bool, Bool), (Any.Type?, Any.Type?), (COpaquePointer, COpaquePointer), (Character, Character), (UInt8, UInt8), (Int8, Int8), (UInt16, UInt16), (Int16, Int16), (UInt32, UInt32), (Int32, Int32), (UInt64, UInt64), (Int64, Int64), (UInt, UInt), (Int, Int), (Float, Float), (Double, Double), (Float80, Float80), (ObjectIdentifier, ObjectIdentifier), (String, String), (Index, Index), (String.UnicodeScalarView.Index, String.UnicodeScalarView.Index), (String.UTF16View.Index, String.UTF16View.Index), (String.UTF8View.Index, String.UTF8View.Index), (UnicodeScalar, UnicodeScalar), (_SwiftNSOperatingSystemVersion, _SwiftNSOperatingSystemVersion), (Bit, Bit), (AnyForwardIndex, AnyForwardIndex), (AnyBidirectionalIndex, AnyBidirectionalIndex), (AnyRandomAccessIndex, AnyRandomAccessIndex), (AccessLevel, AccessLevel), (ContextualKeywordType, ContextualKeywordType), (KeywordType, KeywordType), (PunctuatorType, PunctuatorType), (Token, Token), (ContiguousArray<Element>, ContiguousArray<Element>), (ArraySlice<Element>, ArraySlice<Element>), (Array<Element>, Array<Element>), (AutoreleasingUnsafeMutablePointer<Memory>, AutoreleasingUnsafeMutablePointer<Memory>), (T, T), (LazyFilterIndex<Base>, LazyFilterIndex<Base>), (FlattenCollectionIndex<BaseElements>, FlattenCollectionIndex<BaseElements>), (FlattenBidirectionalCollectionIndex<BaseElements>, FlattenBidirectionalCollectionIndex<BaseElements>), (Set<Element>, Set<Element>), ([Key : Value], [Key : Value]), (SetIndex<Element>, SetIndex<Element>), (DictionaryIndex<Key, Value>, DictionaryIndex<Key, Value>), (_HeapBuffer<Value, Element>, _HeapBuffer<Value, Element>), (HalfOpenInterval<Bound>, HalfOpenInterval<Bound>), (ClosedInterval<Bound>, ClosedInterval<Bound>), (ManagedBufferPointer<Value, Element>, ManagedBufferPointer<Value, Element>), (T?, T?), (T?, _OptionalNilComparisonType), (_OptionalNilComparisonType, T?), (Range<Element>, Range<Element>), (ReverseIndex<Base>, ReverseIndex<Base>), (UnsafeMutablePointer<Memory>, UnsafeMutablePointer<Memory>), (UnsafePointer<Memory>, UnsafePointer<Memory>), ((A, B), (A, B)), ((A, B, C), (A, B, C)), ((A, B, C, D), (A, B, C, D)), ((A, B, C, D, E), (A, B, C, D, E)), ((A, B, C, D, E, F), (A, B, C, D, E, F)), (Self, Self), (E, E.ValueType)
                    try expect(Mirror(reflecting: node.type).subjectType) == testType
                                                                          ^
    make: *** [.build/debug/test_runner] Error 1
    

    Tests could be even more readable if we could write something like try expect(anObject).to.beTypeOf(AClass) and try expect(anObject).to.beProtocolOf(AProtocol).

    Reviewed by ryuichis at 2016-01-17 00:21
  • 11. Automatically run

    By adding an atexit hook when instantiating the global context, we can remove the need to call run manually.

    There may be a better, more flexible solution to the problem, but I figured I'd throw the idea out there!

    Reviewed by stephencelis at 2015-10-10 17:42
  • 12. Spectres should time how long each test case takes and provide user with total time.

    Similiar to mocha and other testing frameworks which allow you to see how long a test case takes. There should also be options to cause test failure when any test goes over some threshold. Mocha default reporter colours the time yellow/red to show slower tests.

    Unfortunately this likely means we have to change the design of the reporters themselves.

    Reviewed by kylef at 2018-08-21 14:21
  • 13. Testing async methods

    Hi, @kylef

    I want to test some async methods. Is it now supported in Spectre?

    I just had a brief look at Spectre but didn't find this feature. Maybe something like this in other BDD testing framework:

    $0.it("should do something asynchronously") {
        waitUntil { done in
            //.. async work
            done()
        }
    }
    

    If this is not yet in Spectre, I'd like to try to implement (although I have no experience on creating a test framework). Before I could get into it, I think it's better to confirm it. What do you think about it?

    Reviewed by onevcat at 2017-03-08 01:52
Related tags
A light-weight TDD / BDD framework for Objective-C & Cocoa
A light-weight TDD / BDD framework for Objective-C & Cocoa

Specta A light-weight TDD / BDD framework for Objective-C. FEATURES An Objective-C RSpec-like BDD DSL Quick and easy set up Built on top of XCTest Exc

Jun 15, 2022
TestKit has been upgraded to a full solution for implementing Behavior-Driven Development (BDD) in Swift iOS apps.

The easiest way to implement full BDD in your Swift iOS projects! Use plain English specs (Gherkin) to drive tests that include both UI automation and interacting with application data & state.

Apr 21, 2022
Simple BDD for iOS

Kiwi: Simple BDD for iOS Kiwi is a Behavior Driven Development library for iOS development. The goal is to provide a BDD library that is exquisitely s

Jun 16, 2022
Test-To-Do-List - Test To Do List with core data
Test-To-Do-List - Test To Do List with core data

test-To-Do-List This is my first pet project with core data Launch screen Main s

Feb 26, 2022
Kfm-ios-test - Test online for iOS Developer position in Kimia Farma or PT. Buana Varia Komputama
Kfm-ios-test - Test online for iOS Developer position in Kimia Farma or PT. Buana Varia Komputama

kfm-ios-test Kimia Farma Mobile iOS Test Test online for iOS Developer position

Feb 23, 2022
A Mac and iOS Playgrounds Unit Testing library based on Nimble.
A Mac and iOS Playgrounds Unit Testing library based on Nimble.

Spry Spry is a Swift Playgrounds Unit Testing library based on Nimble. The best thing about Spry is that the API matches Nimble perfectly. Which means

Jun 5, 2022
PlaygroundTester enables you to easily run tests for your iPad Playgrounds 4 project.

PlaygroundTester PlaygroundTester is a package that enables you to add tests to your iPad Swift Playgrounds project. Installation Just add PlaygroundT

Jun 21, 2022
iOS UI Automation Test Framework

Deprecation: EarlGrey 1.0 is deprecated in favor of EarlGrey 2.0 which integrates it with XCUITest. Please look at the earlgrey2 branch. EarlGrey 1.0

Jun 25, 2022
This repository accompanies Test-Driven Development in Swift: Compile Better Code with XCTest and TDD
This repository accompanies Test-Driven Development in Swift: Compile Better Code with XCTest and TDD

Apress Source Code This repository accompanies Test-Driven Development in Swift: Compile Better Code with XCTest and TDD by Gio Lodi (Apress, 2021). D

May 23, 2022
Test task application based on Swift using CoreData, Alamofire, AlamofireImage and CocoaPods
Test task application based on Swift using CoreData, Alamofire, AlamofireImage and CocoaPods

iTunes Search Test task application based on Swift using CoreData, Alamofire, AlamofireImage and CocoaPods Features ?? Searching music albums by name

Oct 31, 2021
The XCTest Project, A Swift core library for providing unit test support

XCTest The XCTest library is designed to provide a common framework for writing unit tests in Swift, for Swift packages and applications. This version

Jun 15, 2022
Test Library for Swift's Error Handling

CatchingFire CatchingFire is a Swift test framework, which helps making expectations against the error handling of your code. It provides for this pur

May 16, 2022
XCTestExtensions is a Swift extension that provides convenient assertions for writing Unit Test.
XCTestExtensions is a Swift extension that provides convenient assertions for writing Unit Test.

XCTestExtensions Features XCTAssertEventually (that convenient assertions for writing Unit Test). Use "XCTAssertEventually", you can write asynchronou

Feb 28, 2022
Test Flickr API for swift

FlickrSearch Main Features Search and display Flickr photos Display full size photos Zoom in and Zoom out a photo Main Goals: Interact with RESTful Fl

Dec 11, 2021
A Swift test double library. Guava - looks like an apple but it's not.

Guava Guava helps you to make your unit tests more flexible. It allows you to replace parts of your system under test with a test double objects. Tabl

Mar 22, 2022
Trust - Swift DCI Test Library

Trust Swift - DCI Pattern Test Library Support SPM How to use ? struct User {

Feb 13, 2022
Stub your network requests easily! Test your apps with fake network data and custom response time, response code and headers!
Stub your network requests easily! Test your apps with fake network data and custom response time, response code and headers!

OHHTTPStubs OHHTTPStubs is a library designed to stub your network requests very easily. It can help you: test your apps with fake network data (stubb

Jun 23, 2022
I built this application with unit testing and test-driven development to understand TDD theory and practice
I built this application with unit testing and test-driven development to understand TDD theory and practice

TestDrivenDevelopment Description I built this application with unit testing and test-driven development to understand TDD theory and practice, to wri

Dec 21, 2021
test ios UnitTest and UITest

github_actions Bundlerの導入 fastlaneやiOSパッケージマネージャであるCocoaPodsはRubyのライブラリ 開発チームで使用するバージョンを揃えるためにBundlerを導入する bundlerのインストール gem install bundler Gemfile

Nov 3, 2021