BDD Framework and test runner for Swift projects and playgrounds

Related tags

Testing Spectre
Overview

Spectre

Build Status

Special Executive for Command-line Test Running and Execution.

A behavior-driven development (BDD) framework and test runner for Swift projects and playgrounds. It's compatible with both OS X and Linux.

Usage

describe("a person") {
  let person = Person(name: "Kyle")

  $0.it("has a name") {
    try expect(person.name) == "Kyle"
  }

  $0.it("returns the name as description") {
    try expect(person.description) == "Kyle"
  }
}

Reporters

Spectre currently has two built-in reporters, Standard and the Dot reporter. Custom reporters are supported, make a type that conforms to Reporter.

The default reporter can be configured via an environment variable. For example:

$ env SPECTRE_REPORTER=dot swift test
$ env SPECTRE_REPORTER=tap swift test

Standard

The standard reporter produces output as follows:

Passing Tests

Standard Reporter Success

Failing Tests

Standard Reporter Failure

Dot

Using the -t argument, you can use the dot reporter.

Passing Tests

Dot Reporter Success

Failing Tests

Dot Reporter Failure

Expectation

Equivalence

try expect(name) == "Kyle"
try expect(name) != "Kyle"

Truthiness

try expect(alive).to.beTrue()
try expect(alive).to.beFalse()
try expect(alive).to.beNil()

Error handling

try expect(try write()).toThrow()
try expect(try write()).toThrow(FileError.NoPermission)

Comparable

try expect(5) > 2
try expect(5) >= 2
try expect(5) < 10
try expect(5) <= 10

Types

try expect("kyle").to.beOfType(String.self)

Causing a failure

throw failure("Everything is broken.")

Custom assertions

You can easily provide your own assertions, you just need to throw a failure when the assertion does not meet expectaions.

Examples

The following projects use Spectre:

Installation / Running

Swift Package Manager

Check out Commander as an example.

Playground

You can use Spectre in an Xcode Playground, open Spectre.playground in this repository, failures are printed in the console.

Spectre in an Xcode Playground

Comments
  • Xcode reports

    Xcode reports

    Resolves #2

    This PR adds a very naive support for reporting test failures in Xcode. Instead of using runtime to create tests, as suggested in #17 I've added a method XCTestCase.describe that will store a reference to this test case in custom reporter, to use for reporting later, calls global describe function with passed in closure and then runs all the tests. At the end all added tests are removed so that they are not run again later. All individual test failures will appear in Issues navigator.

    Tests will be still run at the application exit (without reporting via Xcode) but if they were already run with XCTestCase.describe they will be removed from the global context and so will not be run twice.

    Also added option to disable colours in output which make output less readable in Xcode console, that does not support ANSI colors anyway.

    screen shot 2017-12-25 at 17 45 57

    opened by ilyapuchka 6
  • Xcode12 support

    Xcode12 support

    XCTestCase.recordFailure(..) method has been deprecated on Xcode12+ and it will cause compile error on Xcode12.

    スクリーンショット 2020-07-05 0 41 33

    To avoid that, added @available block. In the future, XCTestCase. recordFailure(..) should be replaced with new API, XCTestCase.record(_:).

    Video https://developer.apple.com/videos/play/wwdc2020/10687/ Document https://developer.apple.com/documentation/xctest/xctestcase/3546549-record

    opened by kenmaz 4
  • Invalid Exclude '…/SourcePackages/checkouts/Spectre/Sources/Spectre/XCTest@4.swift': File not found.

    Invalid Exclude '…/SourcePackages/checkouts/Spectre/Sources/Spectre/[email protected]': File not found.

    Some dependency of mine has Spectre as a dependency, and when Xcode 13 beta resolves the dependencies, it issues this warning:

    Invalid Exclude '…/SourcePackages/checkouts/Spectre/Sources/Spectre/[email protected]': File not found.
    

    I'm not sure what it means, but this seemed like a reasonable place to report the issue.

    opened by JetForMe 3
  • Build XCTest extensions on linux

    Build XCTest extensions on linux

    Nowadays, swift-corelibs-xctest exists, so it should be possible to use Spectre from XCTest on Linux. This change removes the #if guards around XCTest extensions, with the exception of some Xcode 12+ APIs that have not yet made it upstream.

    Testing done

    Builds and tests on a linux machine using the official Swift docker images:

    $ docker pull swift
    $ docker run -itv $PWD:/code swift swift test --package-path /code
    
    opened by elliottwilliams 3
  • Bump the 0.8.0 to avoid problems with missed files in the playground

    Bump the 0.8.0 to avoid problems with missed files in the playground

    Playground sources a still not available when using 0.8.0 as a dependency. More - various samples (like https://github.com/IBM/FoodTrackerBackend) cannot be correctly configured when using the Swift PM in the CLion (we are currently using CMake for integrating it and CMake complains about files missed). Please, bump the version so the most recent sources are included.

    opened by yeswolf 3
  • More expectations

    More expectations

    • beNotNil to validate that value is not nil
    • notThrow to validate that function does not throw
    • throw(_ match: (Error) -> Bool) to validate that function throws error using custom condition rather than Equatable
    • throw and throw(_:) expectations to better match other APIs (where to is omitted)
    • expect(...).to.not for not expectations, expect(...).to has now better scoped methods and code completion
    opened by ilyapuchka 2
  • Fix build errors when compiling with Xcode 9 GM

    Fix build errors when compiling with Xcode 9 GM

    When building with Xcode 9 GM, we get the following errors:

    [ehyche@GM19403 Spectre (ehyche_fix_swift4)]$ swift build
    Compile Swift Module 'Spectre' (8 sources)
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:98:22: warning: redundant conformance constraint 'Key': 'Equatable'
    public func == <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
                         ^
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:98:57: note: conformance constraint 'Key': 'Equatable' inferred from type here
    public func == <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
                                                            ^
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:108:22: warning: redundant conformance constraint 'Key': 'Equatable'
    public func != <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
                         ^
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:108:57: note: conformance constraint 'Key': 'Equatable' inferred from type here
    public func != <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
                                                            ^
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:98:22: warning: redundant conformance constraint 'Key': 'Equatable'
    public func == <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
                         ^
    /Users/ehyche/src/personal/Spectre/Sources/Expectation.swift:98:57: note: conformance constraint 'Key': 'Equatable' inferred from type here
    public func == <Key: Equatable, Value: Equatable> (lhs: Expectation<[Key: Value]>, rhs: [Key: Value]) throws {
    
    

    These changes fix those build errors. I also verified that it still builds against Swift 3.2 in Xcode 8.

    opened by ehyche 2
  • Updates for future versions of Swift

    Updates for future versions of Swift

    Few last Swift development snapshots are yapping about this:

    ./Spectre/Sources/Expectation.swift:70:25: warning: string interpolation produces a debug description for an optional value; did you mean to make this explicit?
        throw lhs.failure("\(value) is equal to \(rhs)")
                            ^~~~~~~
    ./Spectre/Sources/Expectation.swift:70:26: note: use 'String(describing:)' to silence this warning
        throw lhs.failure("\(value) is equal to \(rhs)")
                            ~^~~~~~
                             String(describing:  )
    ./Spectre/Sources/Expectation.swift:70:26: note: provide a default value to avoid this warning
        throw lhs.failure("\(value) is equal to \(rhs)")
                            ~^~~~~~
                                   ?? <#default value#>
    

    This PR makes Swift a good tail-waggling boy... :wink: All builds and tests passed.

    Swift 3.0
    [x] - Build
    [x] - Test
    Swift 3.0.1
    [x] - Build
    [x] - Test
    Swift DEV SNAPSHOT 2016-11-11
    [x] - Build
    [x] - Test
    

    @kylef Please review and merge. If you do, please bump the Spectre version tag as well. Thanks.

    opened by rbukovansky 2
  • Ability to start tests from Xcode

    Ability to start tests from Xcode

    All of Spectre tests added to global contexts start in atexit. XCode always says that they are passed. As really Spectre tests has not started yet.

    If we start them manually with run. XCode also fails, as inside run exit is called.

    Is there any plan to add ability testing with XCode?

    enhancement help wanted 
    opened by Igor-Palaguta 2
  • Support type comparison

    Support type comparison

    First of all, thank you for working on this, very neat tool.

    I just tried to compare the object type with my expectation. I got it to work with

    https://github.com/ryuichis/swift-ast/blob/66bb167818d4ecc5b009f4f19085826e861d37f5/Tests/declaration/ParsingTypeAliasDeclarationSpec.swift#L119

    However, if I change the line to

    try expect(Mirror(reflecting: node.type).subjectType) == testType

    then I got the following error:

    Tests/declaration/ParsingTypeAliasDeclarationSpec.swift:119:71: error: binary operator '==' cannot be applied to operands of type 'Expectation<Any.Type>' (aka 'Expectation<protocol<>.Type>') and 'Type.Type'
                    try expect(Mirror(reflecting: node.type).subjectType) == testType
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~
    Tests/declaration/ParsingTypeAliasDeclarationSpec.swift:119:71: note: overloads for '==' exist with these partially matching parameter lists: (FloatingPointClassification, FloatingPointClassification), (_MirrorDisposition, _MirrorDisposition), (Mirror.DisplayStyle, Mirror.DisplayStyle), (Bool, Bool), (Any.Type?, Any.Type?), (COpaquePointer, COpaquePointer), (Character, Character), (UInt8, UInt8), (Int8, Int8), (UInt16, UInt16), (Int16, Int16), (UInt32, UInt32), (Int32, Int32), (UInt64, UInt64), (Int64, Int64), (UInt, UInt), (Int, Int), (Float, Float), (Double, Double), (Float80, Float80), (ObjectIdentifier, ObjectIdentifier), (String, String), (Index, Index), (String.UnicodeScalarView.Index, String.UnicodeScalarView.Index), (String.UTF16View.Index, String.UTF16View.Index), (String.UTF8View.Index, String.UTF8View.Index), (UnicodeScalar, UnicodeScalar), (_SwiftNSOperatingSystemVersion, _SwiftNSOperatingSystemVersion), (Bit, Bit), (AnyForwardIndex, AnyForwardIndex), (AnyBidirectionalIndex, AnyBidirectionalIndex), (AnyRandomAccessIndex, AnyRandomAccessIndex), (AccessLevel, AccessLevel), (ContextualKeywordType, ContextualKeywordType), (KeywordType, KeywordType), (PunctuatorType, PunctuatorType), (Token, Token), (ContiguousArray<Element>, ContiguousArray<Element>), (ArraySlice<Element>, ArraySlice<Element>), (Array<Element>, Array<Element>), (AutoreleasingUnsafeMutablePointer<Memory>, AutoreleasingUnsafeMutablePointer<Memory>), (T, T), (LazyFilterIndex<Base>, LazyFilterIndex<Base>), (FlattenCollectionIndex<BaseElements>, FlattenCollectionIndex<BaseElements>), (FlattenBidirectionalCollectionIndex<BaseElements>, FlattenBidirectionalCollectionIndex<BaseElements>), (Set<Element>, Set<Element>), ([Key : Value], [Key : Value]), (SetIndex<Element>, SetIndex<Element>), (DictionaryIndex<Key, Value>, DictionaryIndex<Key, Value>), (_HeapBuffer<Value, Element>, _HeapBuffer<Value, Element>), (HalfOpenInterval<Bound>, HalfOpenInterval<Bound>), (ClosedInterval<Bound>, ClosedInterval<Bound>), (ManagedBufferPointer<Value, Element>, ManagedBufferPointer<Value, Element>), (T?, T?), (T?, _OptionalNilComparisonType), (_OptionalNilComparisonType, T?), (Range<Element>, Range<Element>), (ReverseIndex<Base>, ReverseIndex<Base>), (UnsafeMutablePointer<Memory>, UnsafeMutablePointer<Memory>), (UnsafePointer<Memory>, UnsafePointer<Memory>), ((A, B), (A, B)), ((A, B, C), (A, B, C)), ((A, B, C, D), (A, B, C, D)), ((A, B, C, D, E), (A, B, C, D, E)), ((A, B, C, D, E, F), (A, B, C, D, E, F)), (Self, Self), (E, E.ValueType)
                    try expect(Mirror(reflecting: node.type).subjectType) == testType
                                                                          ^
    make: *** [.build/debug/test_runner] Error 1
    

    Tests could be even more readable if we could write something like try expect(anObject).to.beTypeOf(AClass) and try expect(anObject).to.beProtocolOf(AProtocol).

    enhancement help wanted 
    opened by ryuichis 2
  • Automatically run

    Automatically run

    By adding an atexit hook when instantiating the global context, we can remove the need to call run manually.

    There may be a better, more flexible solution to the problem, but I figured I'd throw the idea out there!

    opened by stephencelis 2
  • Spectres should time how long each test case takes and provide user with total time.

    Spectres should time how long each test case takes and provide user with total time.

    Similiar to mocha and other testing frameworks which allow you to see how long a test case takes. There should also be options to cause test failure when any test goes over some threshold. Mocha default reporter colours the time yellow/red to show slower tests.

    Unfortunately this likely means we have to change the design of the reporters themselves.

    enhancement 
    opened by kylef 0
  • Testing async methods

    Testing async methods

    Hi, @kylef

    I want to test some async methods. Is it now supported in Spectre?

    I just had a brief look at Spectre but didn't find this feature. Maybe something like this in other BDD testing framework:

    $0.it("should do something asynchronously") {
        waitUntil { done in
            //.. async work
            done()
        }
    }
    

    If this is not yet in Spectre, I'd like to try to implement (although I have no experience on creating a test framework). Before I could get into it, I think it's better to confirm it. What do you think about it?

    enhancement 
    opened by onevcat 2
Releases(0.10.1)
  • 0.10.1(Aug 31, 2021)

  • 0.10.0(May 15, 2021)

    Breaking Changes

    • Support for Swift < 4.2 has been dropped.

    Enhancements

    • Reporter type can be set via an environment variable. For example, to use dot reporter:

      $ env SPECTRE_REPORTER=dot swift test
      
    • Additional arguments and options can be passed to Spectre using the SPECTRE_ADDOPTS environment variable, for example:

      $ SPECTRE_ADDOPTS=Tests/SpectreTests/FailureSpec.swift swift test
      
    • Spectre can be passed a set of files to filter which tests will be executed.

    • Add support for Xcode 12.5.

    Source code(tar.gz)
    Source code(zip)
  • 0.9.2(Nov 18, 2020)

    Enhancements

    Bug Fixes

    • Compatibility with some versions of Xcode greater than 12.0.1 where a build error with incompatibility between XCTIssue and XCTIssueReference may be presented with Swift 5.3.
    Source code(tar.gz)
    Source code(zip)
  • 0.9.1(Aug 16, 2020)

  • 0.9.0(Sep 10, 2018)

    Breaking

    • Using Spectre in Xcode has be re-hauled, there are now describe and it methods on XCTestCase which can be used. When used, these tests will be ran directly and reported as XCTest failures and therefore shown in Xcode and Xcode sidebar as XCTest failures.

      Use of the global test context, i.e, global describe and it is no longer permitted when using Spectre with XCTest.

    Enhancements

    • Adds support for Swift 4.2.

    • Unhandled errors will now be reported from the invoked cases source map.

    Source code(tar.gz)
    Source code(zip)
Owner
Kyle Fuller
Kyle Fuller
A light-weight TDD / BDD framework for Objective-C & Cocoa

Specta A light-weight TDD / BDD framework for Objective-C. FEATURES An Objective-C RSpec-like BDD DSL Quick and easy set up Built on top of XCTest Exc

Specta / Expecta 2.3k Dec 20, 2022
TestKit has been upgraded to a full solution for implementing Behavior-Driven Development (BDD) in Swift iOS apps.

The easiest way to implement full BDD in your Swift iOS projects! Use plain English specs (Gherkin) to drive tests that include both UI automation and interacting with application data & state.

Daniel Hall 11 Sep 14, 2022
Simple BDD for iOS

Kiwi: Simple BDD for iOS Kiwi is a Behavior Driven Development library for iOS development. The goal is to provide a BDD library that is exquisitely s

Kiwi 4.1k Dec 22, 2022
Test-To-Do-List - Test To Do List with core data

test-To-Do-List This is my first pet project with core data Launch screen Main s

Artem 0 Feb 26, 2022
Kfm-ios-test - Test online for iOS Developer position in Kimia Farma or PT. Buana Varia Komputama

kfm-ios-test Kimia Farma Mobile iOS Test Test online for iOS Developer position

Erwindo Sianipar 3 Feb 23, 2022
A Mac and iOS Playgrounds Unit Testing library based on Nimble.

Spry Spry is a Swift Playgrounds Unit Testing library based on Nimble. The best thing about Spry is that the API matches Nimble perfectly. Which means

Quick 327 Jul 24, 2022
PlaygroundTester enables you to easily run tests for your iPad Playgrounds 4 project.

PlaygroundTester PlaygroundTester is a package that enables you to add tests to your iPad Swift Playgrounds project. Installation Just add PlaygroundT

Paweł Łopusiński 36 Dec 13, 2022
iOS UI Automation Test Framework

Deprecation: EarlGrey 1.0 is deprecated in favor of EarlGrey 2.0 which integrates it with XCUITest. Please look at the earlgrey2 branch. EarlGrey 1.0

Google 5.5k Dec 30, 2022
This repository accompanies Test-Driven Development in Swift: Compile Better Code with XCTest and TDD

Apress Source Code This repository accompanies Test-Driven Development in Swift: Compile Better Code with XCTest and TDD by Gio Lodi (Apress, 2021). D

Apress 57 Jan 1, 2023
Test task application based on Swift using CoreData, Alamofire, AlamofireImage and CocoaPods

iTunes Search Test task application based on Swift using CoreData, Alamofire, AlamofireImage and CocoaPods Features ?? Searching music albums by name

Alexander Zhukov 0 Oct 31, 2021
The XCTest Project, A Swift core library for providing unit test support

XCTest The XCTest library is designed to provide a common framework for writing unit tests in Swift, for Swift packages and applications. This version

Apple 1k Jan 4, 2023
Test Library for Swift's Error Handling

CatchingFire CatchingFire is a Swift test framework, which helps making expectations against the error handling of your code. It provides for this pur

Marius Rackwitz 75 May 16, 2022
XCTestExtensions is a Swift extension that provides convenient assertions for writing Unit Test.

XCTestExtensions Features XCTAssertEventually (that convenient assertions for writing Unit Test). Use "XCTAssertEventually", you can write asynchronou

shindyu 22 Dec 1, 2022
Test Flickr API for swift

FlickrSearch Main Features Search and display Flickr photos Display full size photos Zoom in and Zoom out a photo Main Goals: Interact with RESTful Fl

Patrick Lee 0 Dec 11, 2021
A Swift test double library. Guava - looks like an apple but it's not.

Guava Guava helps you to make your unit tests more flexible. It allows you to replace parts of your system under test with a test double objects. Tabl

Denis Chagin 10 Mar 22, 2022
Trust - Swift DCI Test Library

Trust Swift - DCI Pattern Test Library Support SPM How to use ? struct User {

gitaek lee 1 Feb 13, 2022
Stub your network requests easily! Test your apps with fake network data and custom response time, response code and headers!

OHHTTPStubs OHHTTPStubs is a library designed to stub your network requests very easily. It can help you: test your apps with fake network data (stubb

Olivier Halligon 4.9k Dec 29, 2022
I built this application with unit testing and test-driven development to understand TDD theory and practice

TestDrivenDevelopment Description I built this application with unit testing and test-driven development to understand TDD theory and practice, to wri

null 1 Dec 21, 2021
test ios UnitTest and UITest

github_actions Bundlerの導入 fastlaneやiOSパッケージマネージャであるCocoaPodsはRubyのライブラリ 開発チームで使用するバージョンを揃えるためにBundlerを導入する bundlerのインストール gem install bundler Gemfile

SUGY 0 Nov 3, 2021