PlaygroundTester enables you to easily run tests for your iPad Playgrounds 4 project.

Last update: May 19, 2022

Swift             License: Unlicense

PlaygroundTester

PlaygroundTester is a package that enables you to add tests to your iPad Swift Playgrounds project.

Installation

Just add PlaygroundTester package to the project as you normally would.

Usage

Adding tests

For PlaygroundTester to find and properly execute tests your test class needs to :

  1. Inherit from TestCase
  2. Be marked as having @objcMembers
    1. Alternatively you can mark each method you want to be discoverable as @objc

At this momment inheriting from another test class is not supported (so you cannot create a class BaseTests: TestCase that you will then inherit other test classes from).

Sample test class declaration :

@objcMembers
final class MyTests: TestCase {
}

setUp / tearDown

You can override four methods to help with setting up and cleaning up after your tests.

  // Called once for the entire test class, before any tests are run.
  open class func setUp() {  }

  // Called before each test method.
  // If this method throws, the test method won't be executed, but `tearDown` will be.
  open func setUp() throws { }

  // Called after each test method.
  open func tearDown() throws { }

  // Called once for the entire test class, after all tests are run.
  open class func tearDown() { }
}

Adding test methods

For PlaygroundTester to discover your test methods and run them automatically they have to :

  1. Be non-private (so public or internal)
  2. Begin with test

Any private methods or ones not begininning with test will not be automatically executed, so you can use this opportunity to define helper methods.

Sample method definition :

func testSample() { }

// Methods that won't be run automaticaly
private func testPrivateSample() { }

func helperMethod() { }

Throwing test methods

Throwing test methods are supported by PlaygroundTester - just define your method following the rules above and add throws to its definition. PlaygroundTester will catch the thrown error and report it.

Sample throwing test method definition :

func testSampleThrowing() throws { }

Asserting

Currently there is a basic set of assertion methods available that mimick the asserting style of XCTest:

// Assert of a passed boolean value is `true`/`false`
public func Assert(_ value: Bool, message: String = "")
public func AssertFalse(_ value: Bool, message: String = "")

// Assert if two passed values are equal / not equal.
public func AssertEqual<T: Equatable>(_ value: T, other: T, message: String = "") 
public func AssertNotEqual<T: Equatable>(_ value: T, other: T, message: String = "")

// assert if passed optional value is `nil` / not `nil`.
public func AssertNil<T>(_ value: T?, message: String = "")
public func AssertNotNil<T>(_ value: T?, message: String = "")

There are a few methods missing from achieving parity with XCTAssert family of methods which will be added later.

Unwrapping

PlaygroundTester provides a similar method to XCTUnwrap:

// Return an unwrapped value, or throw an error if `nil` was passed.
public func AssertUnwrap<T>(_ value: T?, message: String = "") throws -> T

You should mark your test method with throws to avoid the need to handle this thrown error yourself.

Sample method with AssertUnwrap :

func testSampleAssertUnwrap() throws {

  let sampleArray = ["first", "second"]
  
  let firstValue = try XCTUnwrap(sampleArray.first, "Array should have a first element").

  // `firstValue` is non-optional here
}

Expectations

PlaygroundTester supports waiting on expectations to test asynchronous code. Expectations can be configured with 3 properties :

  1. expectedFulfilmentCount (default == 1) - how many times should the expectation be fullfilled to be considered as met. Expectations will fail if they are overfulfilled.
  2. inverted (default == false) - if the expectation is inverted it will fail, if it is fullfilled.
    1. If you have an inverted expectation with expectedFulfilmentCount > 1 it will be considered as met if it gets fullfilled less than expectedFulfilmentCount times.

You use the AssertExpectations method to wait on created expectations :

// Will wait for `timeout` seconds for `expectations` to be fulfilled before continuing test execution.
public func AssertExpectations(_ expectations: [Expectation], timeout: TimeInterval)

Sample test with expectation :

func testSampleExpectation() {
  let expectation = Expectation(name: "Wait for main thread")
  
  DispatchQueue.main.async { 
    expectation.fulfill()
  }
  
  AssertExpectations([expectation], timeout: 2)
}

At this moment unwaited expectations don't trigger an assertion failure.

Runing tests

In order to execute your tests you need to do one final thing : exchange your app view for PlaygroundTesterView. In your App object just do this :

struct Myapp: App {
    var body: some Scene {
        WindowGroup {
          // YourContentView()
          PlaygroundTester.PlaygroundTesterView()
        }
    }
}

After that when running the app either in fullscreen or in preview mode will instead discover and run your tests.

Inspecting results

After the tests are run, you can navigate them to inspect their results and see which assertions failed.

pg_tester_sample.mp4

Patching Package.swift

Swift Playgrounds doesn't support multiple targets at this time, so your test files will need to be kept alongside regular application code. The package itself has compilation guards around its code, so that when creating a release build most of it will be omitted from you production app. What is left is the minimal set of object and function definitions, so that your code compiles just fine, and all calls to PlaygroundTester provided objects/functions resolves to basically no-ops.

If you'd like to also discard your tests from release builds, you'll need to add a compilation flag to your app.

For now you'll need to follow these steps to add a compilation flag to your project :

  1. Send the app project to a Mac (for example via AirDrop)
  2. Open the package contents (right click -> Show Package Contents)
  3. Open Package.swift file
  4. This file should contain a single .executableTarget definition.
  5. Add this argument to the target : swiftSettings: [.define("TESTING_ENABLED", .when(configuration: .debug))]
  6. Save the file and share the app back to your iPad.

In the end the target definition should look similar to this :

targets: [
        .executableTarget(
            name: "AppModule",
            dependencies: [
              // if any
            ],
            path: ".",
            swiftSettings: [.define("TESTING_ENABLED", .when(configuration: .debug))]
        )
    ]

You can of course choose any name for the flag.

NOTE : I hope to automate this process in Patching Package.swift

Supported features

  • Automatic test discovery & execution
  • UI for inspecting test results
  • Basic assertions
  • Expectations
  • Throwing tests methods
  • setUp/tearDown methods

Roadmap

Things I'd like to explore and add to PlaygroundTester (random order):

  • Automatic Package.swift patching
  • Filtering - by test result and by name
    • Possibly with persistence, to allow quick feedback for TDD
  • Single test method / test suite execution
  • Flat list view
  • Assertion parity with XCTest
  • Explore support for async/await
  • Explore support for Combine
  • ... and possibly more

Please check Issues for more info.


GitHub

https://github.com/Losiowaty/PlaygroundTester
Comments
  • 1. I can't see the text in Dark Mode

    At this line, Color.white is used for the background color. However, the text on it uses the default color (.primary), so I can't see the text in Dark Mode.

    B81D7DE2-669A-4A13-877B-FF26838A93EB

    Reviewed by kkk669 at 2022-03-25 15:29
  • 2. Add filtering button for test failures

    I think it resolves https://github.com/Losiowaty/PlaygroundTester/issues/4?

    The button is added as a bar button item, its state is replicated across both TestSuitesView and TestMethodsView.

    TestSuitesView

    Simulator Screen Shot - iPhone 12 - 2022-02-16 at 19 50 51 Simulator Screen Shot - iPhone 12 - 2022-02-16 at 19 50 53

    TestMethodsView

    Simulator Screen Shot - iPhone 12 - 2022-02-16 at 19 51 13 Simulator Screen Shot - iPhone 12 - 2022-02-16 at 19 51 19

    Let me know if you need anything added/changed here. I tested it manually but it seems like it works, potentially we could add some snapshots with mock results but I didn't want to add another dependency so I'll leave it as is for now.

    I can also grab https://github.com/Losiowaty/PlaygroundTester/issues/5 next if you'd like.

    Reviewed by pawelurb at 2022-02-16 18:56
  • 3. Handle unwaited expectations

    Currently unwaited expectations have no effect. In XCTest those will cause an assertion failure and will be flagged during test execution. I believe this is a good thing to have, so should be added to package.

    Reviewed by Losiowaty at 2022-02-16 00:00
  • 4. Enable running a single test class / method

    Research a possibility to run a single test class / test method only to help with TDD. If running a single class / method proves too tricky, another approach would be to try to have a persistent filter so that only filtered methods are visible.

    Reviewed by Losiowaty at 2022-02-07 09:43
  • 5. Support warnings

    Some situations may not be considered test failures per se, but it may be worth notifying the user that they occured

    • no assertions triggered during test
    • non-strict Expectation was overfulfilled
    Reviewed by Losiowaty at 2022-02-06 09:49
  • 6. Package.swift patcher

    Automate adding DEBUG flag to Package.swift

    • as a shortcut? -- already investigate almost entirely, fails on saving
    • as an Playgrounds project that updates the file?
    • inline, if the package doesn't detect it?
    Reviewed by Losiowaty at 2022-02-01 00:58
Erik is an headless browser based on WebKit. An headless browser allow to run functional tests, to access and manipulate webpages using javascript.
Erik is an headless browser based on WebKit. An headless browser allow to run functional tests, to access and manipulate webpages using javascript.

Erik Erik is a headless browser based on WebKit and HTML parser Kanna. An headless browser allow to run functional tests, to access and manipulate web

May 19, 2022
Switchboard - easy and super light weight A/B testing for your mobile iPhone or android app. This mobile A/B testing framework allows you with minimal servers to run large amounts of mobile users.

Switchboard - easy A/B testing for your mobile app What it does Switchboard is a simple way to remote control your mobile application even after you'v

Apr 28, 2022
ShortWebCore - This iOS library lets you run automations on Web Views.

This iOS library lets you run automations on Web Views. Example (Optional) Declare class conforming to AutomationRunnerDelegate: import S

May 14, 2022
BDD Framework and test runner for Swift projects and playgrounds
BDD Framework and test runner for Swift projects and playgrounds

Spectre Special Executive for Command-line Test Running and Execution. A behavior-driven development (BDD) framework and test runner for Swift project

Apr 20, 2022
A Mac and iOS Playgrounds Unit Testing library based on Nimble.
A Mac and iOS Playgrounds Unit Testing library based on Nimble.

Spry Spry is a Swift Playgrounds Unit Testing library based on Nimble. The best thing about Spry is that the API matches Nimble perfectly. Which means

Apr 16, 2022
Freezer is a library that allows your Swift tests to travel through time by mocking NSDate class.

Freezer Freezer is a library that allows your Swift tests to travel through time by mocking NSDate class. Usage Once Freezer.start() has been invoked,

Jul 17, 2017
AutoMocker is a Swift framework that leverages the type system to let you easily create mocked instances of your data types.

AutoMocker Context AutoMocker is a Swift framework that leverages the type system to let you easily create mocked instances of your data types. Here's

Jan 26, 2022
Mockit is a Tasty mocking framework for unit tests in Swift 5.0
Mockit is a Tasty mocking framework for unit tests in Swift 5.0

Mockit Introduction Mockit is a Tasty mocking framework for unit tests in Swift 5.0. It's at an early stage of development, but its current features a

Apr 26, 2022
Swift Package with examples of how to tests iOS apps

GimmeTheCodeTDD A swift package with examples of unit tests in iOS development. Requirements Xcode 13 Content Dependency Injection Constructor Injecti

Oct 11, 2021
Trying to implement Unit Tests for @Binding properties in a ViewModel

BindingTester Trying to implement Unit Tests for @Binding properties in a ViewModel ViewModel to be tested class SheetViewModel: ObservableObject {

Oct 22, 2021
Library for unifying the approach to network mocking in iOS unit- & UI-tests.

TinkoffMockStrapping Example To run the example project, clone the repo, and run pod install from the Example directory first. Requirements Installati

May 4, 2022
Catching fatal errors in unit tests

Precondition Catching When running tests which hit fatal errors, often preconditions the built-in support with XCTest. One package which supports cach

Nov 28, 2021
Write unit tests which test the layout of a view in multiple configurations
Write unit tests which test the layout of a view in multiple configurations

Overview This library enables you to write unit tests which test the layout of a view in multiple configurations. It tests the view with different dat

Apr 14, 2022
Bluepill is a reliable iOS testing tool that runs UI tests using multiple simulators on a single machine
Bluepill is a reliable iOS testing tool that runs UI tests using multiple simulators on a single machine

Bluepill is a tool to run iOS tests in parallel using multiple simulators. Motivation LinkedIn created Bluepill to run its large iOS test suite in a r

May 11, 2022
UITest helper library for creating readable and maintainable tests

UITestHelper General information When creating UI tests you will see that you are often repeating the same pieces of code. The UITestHelper library wi

Feb 20, 2022
Swift framework containing a set of helpful XCTest extensions for writing UI automation tests
Swift framework containing a set of helpful XCTest extensions for writing UI automation tests

AutoMate • AppBuddy • Templates • ModelGenie AutoMate AutoMate is a Swift framework containing a set of helpful XCTest extensions for writing UI autom

May 8, 2022
Upload failing iOS snapshot tests cases to S3
Upload failing iOS snapshot tests cases to S3

Second Curtain If you're using the cool FBSnapshotTestCase to test your iOS view logic, awesome! Even better if you have continuous integration, like

Mar 13, 2021
Tool for generating Acceptance Tests in Xcode, inspired by Fitnesse
Tool for generating Acceptance Tests in Xcode, inspired by Fitnesse

AcceptanceMark is a tool for generating Acceptance Tests in Xcode, inspired by Fitnesse. Read this blog post for a full introduction to AcceptanceMark

Feb 7, 2022
A collection of useful test helpers designed to ease the burden of writing tests for iOS applications.
A collection of useful test helpers designed to ease the burden of writing tests for iOS applications.

MetovaTestKit is a collection of useful test helpers designed to ease the burden of writing tests for iOS applications. Requirements Installation Usag

Aug 29, 2021