Snapshot view unit tests for iOS

Overview

iOSSnapshotTestCase (previously FBSnapshotTestCase)

Build Status CocoaPods Compatible Carthage compatible Swift Package Manager

What it does

A "snapshot test case" takes a configured UIView or CALayer and uses the necessary UIKit or Core Animation methods to generate an image snapshot of its contents. It compares this snapshot to a "reference image" stored in your source code repository and fails the test if the two images don't match.

Why?

We write a lot of UI code. There are a lot of edge cases that we want to handle correctly when you are creating UIView instances:

  • What if there is more text than can fit in the space available?
  • What if an image doesn't match the size of an image view?
  • What should the highlighted state look like?

It's straightforward to test logic code, but less obvious how you should test views. You can do a lot of rectangle asserts, but these are hard to understand or visualize. Looking at an image diff shows you exactly what changed and how it will look to users.

iOSSnapshotTestCase was developed to make snapshot tests easy.

Installation

Step 1: Add iOSSnapshotTestCase to your project

CocoaPods

Add the following lines to your Podfile:

target "Tests" do
  use_frameworks!
  pod 'iOSSnapshotTestCase'
end

If your test target is Objective-C only use iOSSnapshotTestCase/Core instead, which doesn't contain Swift support.

Carthage

Add the following line to your Cartfile:

github "uber/ios-snapshot-test-case" ~> 8.0.0

Swift Package Manager

Add the following line to your Package.swift:

dependencies: [
  .package(url: "https://github.com/uber/ios-snapshot-test-case.git", from: "8.0.0"),
],

...or integrate with Xcode via File -> Swift Packages -> Add Package Dependency... using the URL of the repository. We recommend using "Up to Next Major" with the Version field, as we use Semantic Versioning and only put breaking changes in major versions.

Step 2: Setup Test Scheme

Replace "Tests" with the name of your test project.

  1. There are three ways of setting reference image directories, the recommended one is to define FB_REFERENCE_IMAGE_DIR in your scheme. This should point to the directory where you want reference images to be stored. We normally use this:
Name Value
FB_REFERENCE_IMAGE_DIR $(SOURCE_ROOT)/$(PROJECT_NAME)Tests/ReferenceImages
IMAGE_DIFF_DIR $(SOURCE_ROOT)/$(PROJECT_NAME)Tests/FailureDiffs

Define the IMAGE_DIFF_DIR to the directory where you want to store diffs of failed snapshots. There are also three ways to set failed image diff directories.

Creating a snapshot test

  1. Subclass FBSnapshotTestCase instead of XCTestCase.
  2. From within your test, use FBSnapshotVerifyView.
  3. Run the test once with self.recordMode = YES; in the test's -setUp method. (This creates the reference images on disk.)
  4. Remove the line enabling record mode and run the test.

Features

  • Automatically names reference images on disk according to test class and selector.
  • Prints a descriptive error message to the console on failure. (Bonus: failure message includes a one-line command to see an image diff if you have Kaleidoscope installed.)
  • Supply an optional "identifier" if you want to perform multiple snapshots in a single test method.
  • Support for CALayer via FBSnapshotVerifyLayer.
  • usesDrawViewHierarchyInRect to handle cases like UIVisualEffect, UIAppearance and Size Classes.
  • fileNameOptions to control appending the device model (iPhone, iPad, iPod Touch, etc), OS version, screen size and screen scale to the images (allowing to have multiple tests for the same «snapshot» for different OSs and devices).

Notes

Your unit tests should be inside an "application" bundle, not a "logic/library" test bundle. (That is, it should be run within the Simulator so that it has access to UIKit.)

However, if you are writing snapshot tests inside a library/framework, you might want to keep your test bundle as a library test bundle without a Test Host.

Read more on this here.

Authors

iOSSnapshotTestCase was written at Facebook by Jonathan Dann with significant contributions by Todd Krabach.

Today it is maintained by Uber.

License

iOSSnapshotTestCase is MIT–licensed. See LICENSE.

Comments
  • Allowing flexible folder names and trimming test words from image names

    Allowing flexible folder names and trimming test words from image names

    Hi,

    As discussed in #8 I took the liberty to try an implementation that would keep the current behaviour while adding two different possibilities:

    • Allow the folderName to be set, which will impact the folder in which the images will be saved.
    • Allow trimming the test prefix of the images names.

    This also impacts the Failure folder.

    Before                            After

    screenshot 2018-02-25 12 22 12        screenshot 2018-02-25 12 13 55

    In my opinion this makes up for better readability, and also helps referencing custom views.

    You can also create subfolders using folderName = "folder/subfolder" which allows you to have:

    screenshot 2018-02-25 12 31 45

    Note: I have almost no experience with objective-C, so if there's anything that might not be in the best way, accept my apologies and please let me know.

    opened by nunogoncalves 34
  • Support Swift Package Manager

    Support Swift Package Manager

    Support Swift Package Manager

    Add support to SPM

    Changed

    • Add Package.swift
    • Project folders organization for compatibility to structure Swift Package Manager
    • Fix Headers Objc
    • Change load resource into test target
    opened by bfernandesbfs 18
  • Color mismatch on iPhone8

    Color mismatch on iPhone8

    Hello,

    I am using your library and I generate the snapshots using xcodebuild in 4 different devices at once (iPhone8, 8Plus, SE and X).

    After generating the 4 snapshots for each test, when I run the tests in xcode, some of them are failing. The diff is clearly not helping: diff_testcountryandprovincestate_iphone_375x667 2x

    Reference image: failed_testcountryandprovincestate_iphone_375x667 2x

    I found this issue but it is not helping and I don't believe is the same problem.

    I have used the Digital Color Meter on the diff image and there are definitely parts of the image where the color differs from one pixel to another. Also, there is a bit of ghosting at the bottom (where the big blue button is).

    This problem only appears on iPhone 8. It has something to do with colors. I have generated the reference images and run the test on the same machine, same simulator.

    opened by victorBaro 14
  • iOSSnapshotTestCase  doesn't work on Bitrise.

    iOSSnapshotTestCase doesn't work on Bitrise.

    Hi Guys,

    I am happy to see iOSSnapshotTestCase still alive. Also, I have seen you fixed a lot of issues. But There is a problem continues integrity system. For example, our team uses Bitrise. Without Bitrise all test methods work fine but with Bitrise I am getting an error from iOSSnapshotTestCase methods . I share below. Rest of the methods work fine with Bitrise. Feel free to share fix step thanks a lot.

    PMITests.LoginViewControllerSnapTest testDefault, failed - Snapshot comparison failed: Optional(Error Domain=FBSnapshotTestControllerErrorDomain Code=1 "Unable to load reference image." UserInfo={NSLocalizedFailureReason=Reference image not found. You need to run the test in record mode, NSLocalizedDescription=Unable to load reference image., FBReferenceImageFilePathKey=/Users/vagrant/Library/Developer/Xcode/DerivedData/Acadia-aashxoxksbzmztdoledkxuavkfly/Build/Products/Integration-iphonesimulator/Vibrent.app/PlugIns/PMITests.xctest/ReferenceImages_64/PMITests.LoginViewControllerSnapTest/[email protected]}) /Users/vagrant/git/Acadia/PMITests/SnapshotTestCases/LoginViewControllerSnapTest.swift:32

            let vc = LoginViewController()
            FBSnapshotVerifyView(vc.view)
    

    testDefault, failed - Snapshot comparison failed: Optional(Error Domain=FBSnapshotTestControllerErrorDomain Code=1 "Unable to load reference image." UserInfo={NSLocalizedFailureReason=Reference image not found. You need to run the test in record mode, NSLocalizedDescription=Unable to load reference image., FBReferenceImageFilePathKey=/Users/vagrant/Library/Developer/Xcode/DerivedData/Acadia-aashxoxksbzmztdoledkxuavkfly/Build/Products/Integration-iphonesimulator/Vibrent.app/PlugIns/PMITests.xctest/ReferenceImages_32/PMITests.LoginViewControllerSnapTest/[email protected]}) /Users/vagrant/git/Acadia/PMITests/SnapshotTestCases/LoginViewControllerSnapTest.swift:32

            let vc = LoginViewController()
            FBSnapshotVerifyView(vc.view)
    
     Executed 24 tests, with 2 failures (0 unexpected) in 0.534 (0.619) seconds
    
    opened by durul 11
  • Trims away the module names on Swift classes

    Trims away the module names on Swift classes

    Port from https://github.com/facebookarchive/ios-snapshot-test-case/pull/219 to new repo.

    What is this PR about:

    Calling NSStringFromClass in Objc for Swift classes creates long strings in form of "Module.Class".

    To be consistent with the Objc style, specially noticeable on reference paths, this patch trims away the module names on Swift classes.

    Important:

    Merging this could cause tests to fail for users that are currently testing Swift classes, since the image references may not be found.

    opened by dzenbot 11
  • Snapshots do not render sublayers

    Snapshots do not render sublayers

    I'm currently trying to integrate this library and have been having issues snapshotting more complex views that include sublayers. The structure of the view includes 2 sublayers on the view's layer to handle more complex shadows. When using this library the shadows do not render when using FBSnapshotVerifyView

    I have a workaround that makes use of iOS 10's UIGraphicsImageRenderer to render the view into an image and then pass that image into a UIImageView and send it along to FBSnapshotVerifyView and this gets the job done. See the attached screenshots as an example.

    Would it be possible to have support for UIGraphicsImageRenderer in order to render and snapshot views that include sublayers?

    Thanks!

    Just using FBSnapshotVerifyView: testdefaultcard 2x

    Using UIGraphicsImageRenderer: testdefaultcard 2x

    opened by rami-a 10
  • Using 'iOSSnapshotTestCase/Core', '2.2.0' but FBSnapshotTestCase does not compile

    Using 'iOSSnapshotTestCase/Core', '2.2.0' but FBSnapshotTestCase does not compile

    We have been using 'FBSnapshotTestCase/Core', '2.1.4' for quite som time and wanted to upgrade to 'iOSSnapshotTestCase/Core', '2.2.0'

    However, the class FBSnapshotTestCase does not compile.

    See screen shot from AppCode.

    image

    opened by cckring 10
  • Not support Swift Package Manager (SPM)

    Not support Swift Package Manager (SPM)

    Add Package to iOS:

    Package Resolution Failed

    "ios-snapshot-test-case" could not be resolved:

    https://github.com/uber/ios-snapshot-test-case has no Package.swift manifest for version 6.0.3

    Screen Shot 2019-06-27 at 17 41 21

    opened by PH9 9
  • Added Tolerance on pixel level

    Added Tolerance on pixel level

    Pixel Tolerance

    The Problem

    Does it really matter if there is 2 shades of colour difference in an element on the screen ? Is it really worth spending our time fixing it ?

    There have been a couple of instances where the snapshot testing has produced entirely grey diff images, or at least seemingly so to the naked eye, when run on different machines. After increasing contrast in an image editor, there are minuscule changes (sometimes a couple of shades of grey difference) in the colour of a pixel. This brings about test failures which are difficult to diagnose, provide very little value and chew up developer time.

    See #27 and #37 for an example of this.

    Regular Tolerance

    The default tolerance implementation in the test case is very simplistic:

    no. of diff. pixels / total pixels = percentage difference

    Under the current implementation introducing even the smallest of tolerances here to ignore a slight difference in colour could result in small actual UI changes being missed. Basically any change to any element with a small enough share of the screens pixels will be missed.

    The Solution

    Pixel Tolerance

    This PR introduces a new tolerance implementation which considers the percentage change in a pixels RGBA colour components to determine whether or not a pixel is considered 'different'.

    (referencePixel.colourComponent - actualPixel.colourComponent) / 256 = percentage difference

    This new implementation means we are able to tolerate slight changes in a pixels colour components but still pick up what would be a 'major UI change' such as moving, adding, or removing an element on the screen or drastically changing its colour.

    The new implementation can be used as a replacement, or work in tandem with the old implementation to provide the greatest flexibility to the developer. It's also fully backwards compatible and shouldn't cause any issues for consumers getting the latest version of ios-snapshot-test-case.

    opened by JerryTheIntern 9
  • Not saving failed snapshots

    Not saving failed snapshots

    I think #44 broke saving failed snapshots if you don't set anything (https://github.com/uber/ios-snapshot-test-case/pull/44/files#diff-00c0098c680647a84df94272d707271eR42).

    Adding _imageDiffDirectory = NSTemporaryDirectory(); on -[FBSnapshotTestController initWithTestClass:] fixes the issue for me, but I'm not sure if it's the "right" fix.

    I'm using 4.0.0.

    opened by marcelofabri 8
  • Users should be able to define a  `manualScale` instead of being locked into the `mainScreen.scale`

    Users should be able to define a `manualScale` instead of being locked into the `mainScreen.scale`

    In order to stay flexible, users of this library "need" to upload two version of there reference images to keep tests passing on various environments (Other Developers, CI, etc)

    This proposal would not lock users in to always using the "one true simulator" to run their tests on.

    We've recently adopted Snapshot testing at my shop and I don't really want to be the the code review czar ensuring every new PR which adds a snapshot has one at @2x and @3x. Other cancers are the size of our git repo growing over time to support 2 screenshots for each new test.

    Currently the framework uses [[UIScreen mainScreen] scale] for saving and comparing snapshots of views and layers.

    If the view or layer are meant to be displayed at a different scale that doesn't match the main screen's one, the screenshots get resized (minified / magnified) and the comparisons are not correct any more from the point of view of the user, since the images are different from what will be shown to the user.

    This PR tries to remove this limitation for those special cases by allowing the test developer to manually override the scale that will be used.


    This work was originally proposed by @djrtl as https://github.com/facebookarchive/ios-snapshot-test-case/pull/166


    Thanks for doing the original implementation @djrtl, I hope I did not overstep by re-opening this here now that this lib is under new ownership.

    opened by regnerjr 8
  • Take multiple snapshots to the same view

    Take multiple snapshots to the same view

    Is it possible to take a snapshot to view1 then move to view2 then go back to view1 to check if any changes happened during that moving between the two views?

    opened by ahmdyasser 0
  • Switch view appearance before verifying

    Switch view appearance before verifying

    I wanted to ask if someone know a way to switch appearance of a view before verifying?

    So far I have tried to set viewController / view.overrideUserInterfaceStyle = .light but it doesn't work.

    opened by elusivestudio 1
  • [Solved] Library not loaded: @rpath/XCTest.framework/XCTest

    [Solved] Library not loaded: @rpath/XCTest.framework/XCTest

    In case you get a SIGABRT when running your app after adding the library, make sure you select the unit test target and not the main app target when adding the package. Captura de Tela 2022-05-12 às 20 19 31

    opened by rafaelclaycon 0
  • Add removesErrorSuffix to make it easier to use with a test that throws

    Add removesErrorSuffix to make it easier to use with a test that throws

    This changes the default behavior, so it's technically a breaking change. However, I think this default behavior makes more sense - and hopefully it won't break things to many people since adding throws to a unit test is relatively new (~1 year)

    Updated version of #145

    opened by marcelofabri 0
  • different images on Intel and Apple M1

    different images on Intel and Apple M1

    I'm seeing some tests with different binary image results on a Mac with Intel and a Mac with Apple M1. The diff image is all grey, so I guess it's just some bits.

    Has anyone also seen this problem?

    opened by kikeenrique 4
Releases(8.0.0)
Owner
Uber Open Source
Open Source Software at Uber
Uber Open Source
Write unit tests which test the layout of a view in multiple configurations

Overview This library enables you to write unit tests which test the layout of a view in multiple configurations. It tests the view with different dat

LinkedIn 565 Nov 16, 2022
Library for unifying the approach to network mocking in iOS unit- & UI-tests.

TinkoffMockStrapping Example To run the example project, clone the repo, and run pod install from the Example directory first. Requirements Installati

Online financial ecosystem 22 Jan 3, 2023
TestSchedulerDemo - Demonstration code for iOS Unit Tests and Asynchronous

TestSchedulerDemo This repository contains demonstration code for my Medium arti

Carsten Wenderdel 0 Mar 19, 2022
Mockit is a Tasty mocking framework for unit tests in Swift 5.0

Mockit Introduction Mockit is a Tasty mocking framework for unit tests in Swift 5.0. It's at an early stage of development, but its current features a

Syed Sabir Salman-Al-Musawi 118 Oct 17, 2022
Trying to implement Unit Tests for @Binding properties in a ViewModel

BindingTester Trying to implement Unit Tests for @Binding properties in a ViewModel ViewModel to be tested class SheetViewModel: ObservableObject {

Raphael Guye 0 Oct 22, 2021
Catching fatal errors in unit tests

Precondition Catching When running tests which hit fatal errors, often preconditions the built-in support with XCTest. One package which supports cach

Brennan Stehling 0 Nov 28, 2021
Marvel - Marvel Characters App using MVVM, and including unit tests

Marvel About The purpose of this project is to develop a app using MVVM, and inc

null 1 Mar 20, 2022
Detailed explanations and implementations of various maths concepts for writing high performance code/algorithms backed with Unit tests.

Detailed explanations and implementations of various maths concepts which can help software Engineers write high performance code/algorithms backed with Unit tests.

Mussa Charles 2 Sep 25, 2022
Snapshot testing tool for iOS and tvOS

SnapshotTest is a simple view testing tool written completely in Swift to aid with development for Apple platforms. It's like unit testing for views.

Pär Strindevall 44 Sep 29, 2022
iOS Simulator type agnostic snapshot testing, built on top of the FBSnapshotTestCase.

SnappyTestCase iOS Simulator type agnostic snapshot testing, built on top of the FBSnapshotTestCase. Why? Snapshot testing helps to deliver views that

Tooploox 15 Oct 11, 2019
📸 Delightful Swift snapshot testing.

?? SnapshotTesting Delightful Swift snapshot testing. Usage Once installed, no additional configuration is required. You can import the SnapshotTestin

Point-Free 3k Jan 3, 2023
A Mac and iOS Playgrounds Unit Testing library based on Nimble.

Spry Spry is a Swift Playgrounds Unit Testing library based on Nimble. The best thing about Spry is that the API matches Nimble perfectly. Which means

Quick 327 Jul 24, 2022
Swift Package with examples of how to tests iOS apps

GimmeTheCodeTDD A swift package with examples of unit tests in iOS development. Requirements Xcode 13 Content Dependency Injection Constructor Injecti

Dominik Hauser 8 Oct 11, 2021
Bluepill is a reliable iOS testing tool that runs UI tests using multiple simulators on a single machine

Bluepill is a tool to run iOS tests in parallel using multiple simulators. Motivation LinkedIn created Bluepill to run its large iOS test suite in a r

Mobile Native Foundation 3.1k Jan 3, 2023
A collection of useful test helpers designed to ease the burden of writing tests for iOS applications.

MetovaTestKit is a collection of useful test helpers designed to ease the burden of writing tests for iOS applications. Requirements Installation Usag

null 23 Aug 29, 2021
The XCTest Project, A Swift core library for providing unit test support

XCTest The XCTest library is designed to provide a common framework for writing unit tests in Swift, for Swift packages and applications. This version

Apple 1k Jan 4, 2023
Runtime introspection and unit testing of SwiftUI views

ViewInspector ??️‍♂️ for SwiftUI ViewInspector is a library for unit testing SwiftUI views. It allows for traversing a view hierarchy at runtime provi

Alexey Naumov 1.5k Jan 8, 2023
XCTestExtensions is a Swift extension that provides convenient assertions for writing Unit Test.

XCTestExtensions Features XCTAssertEventually (that convenient assertions for writing Unit Test). Use "XCTAssertEventually", you can write asynchronou

shindyu 22 Dec 1, 2022
I built this application with unit testing and test-driven development to understand TDD theory and practice

TestDrivenDevelopment Description I built this application with unit testing and test-driven development to understand TDD theory and practice, to wri

null 1 Dec 21, 2021