The name of this repository is "MetovaTestKit", the name of the CocoaPod is "MTK", and the README references the names "Metova Test Kit" and "MTK". This leads to a bit of confusion. Furthermore, "MTK" isn't very descriptive. It's analogous to a bad variable name.

In light of this, I'd like to deprecate "MTK" in favor of "MetovaTestKit". I don't think it's necessary for us to send a PR to CocoaPods' master spec repo—and we wouldn't want to since that would break existing Podfiles that reference MTK. Instead, I believe we can simply rename our podspec, push the new podspec, then run:

pod trunk deprecate MTK --in-favor-of=MetovaTestKit

However, I don't want to just rename MTK. I'd like to make a number of breaking changes to make the framework look and feel like a Swift framework rather than an Objective-C framework—after all, it's written in Swift. So, I think we should drop the "MTK" prefix from all assertions, and name them with lowerCamelCase as per the Swift API Design Guidelines.

I believe we originally went with the all caps "MTK" prefix in order to mirror the naming conventions of the XCTest assertions, but I don't think there's really any value in doing that. After all, all the new methods that have been added to XCTest no longer follow that convention (e.g. expectation(description:)).

The practice of prefixing class/method names with initials came from Objective-C where it was necessary in order to avoid naming conflicts. However, with Swift this isn't a problem. Even if a project was importing MetovaTestKit along with another framework that also happened to have an extension on XCTestCase with a method with the same name as one from MetovaTestKit (e.g. assertControl(_:sends:to:for:)), the naming conflict can be resolved by fully qualifying it with the name of the module:

MetovaTestKit.assertControl(testVC.myControl, sends: #selector(MyViewController.didTapControl(_:)), to: testVC, for: .touchUpInside)

Any thoughts or concerns?

@nhgrif This adds an assertion for testing that a view controller presented a UIAlertController with a particular style, title, message, and actions. Just so you're aware, in order to test this assertion, I had to be able to successfully present an alert which isn't possible without a window. So, I added a new test target that actually runs an empty UIKit app so I have a window to add my presenter view controller to for the tests.

XCTest allows test methods to denote a test failure by marking the test as throwing. This makes the testing of throwing functions a tad simpler.

func testThrowingFunctionSucceeds() throws {
    try doSomethingThatShouldNotFail()
}

This tends to be much more elegant than something like this:

func testThrowingFunctionSucceeds() {
    do {
        try doSomethingThatShouldNotFail()
    }
    catch {
        XCFail("An error occurred: \(error)")
    }
}

In the same light, the test(_:) function could be much more ergonomic by allowing uncaught errors to fail a test via propagation.

func testThrowingMethodSucceeds() throws {
    try Object.test { instance in
        try instance.doSomethingThatShouldNotFail()
    }
}

The test(_:) function was marked rethrows to allow the user to forego this functionality while testing functions that don't throw. This prevents the cruft of having to try non-throwing functionality within a test when unnecessary.

func testNonThrowingMethod() {
    Object.test { instance in
        instance.doSomethingThatCannotFail()
        
        // Make assertions below...
    }
}

This PR is source breaking – anything that is already using the test(_:) method will need to add the throws and rethrows keywords. This should be a minimal change, all of which can be caught by the compiler due to a Type does not conform to protocol 'MTKTestable' compiler error.

@nhgrif I kind of forgot about this for a while. Basically, the day after releasing version 1.1.0 which included the MTKWaitThenContinueTest(after:) function, @schrismartin made a keen observation which makes me want to deprecate this original version and add a new version that takes no closure.

Usage for the original version looks like this:

// Perform some async action
MTKWaitThenContinueTest(after: 1) {
    // Perform assertions
}

What Chris pointed out is that this works just the same because it's a blocking function:

// Perform some async action
MTKWaitThenContinueTest(after: 1) {}
// Perform assertions

I think the benefit of removing the closure is that it simplifies the interface and we'd be able to remove these warnings in the function's docs:

    /// - Warning: This method works by creating an expectation and calling `waitForExpectations(timeout:handler:)`. Because of this, the following actions will result in a failure:
    ///   - Creating nested calls to `MTKWaitThenContinueTest(after:on:testAction:)`.
    ///   - Creating expectations within `testAction` that are not fulfilled within `testAction`
    ///   - Calling `waitForExpectations(timeout:handler:)` within `testAction`.

Technically, we'd still want to include a warning to explain that you won't be able to create an expectation that isn't fulfilled prior to calling the method, but we'll be able to remove that warning for iOS 11 since the expectation API has been updated to allow us to wait only for a specific expectation.

Of course, because we've already released version 1.1.0, we can't just remove the closure argument because it would break backwards compatibility. One backwards compatible option would be to make the closure optional and add a default value of nil. However, this strategy won't allow us to add a deprecation annotation on the function. So, I'm proposing that we leave the original function as is and add a deprecation annotation. Then add a new function that's identical but doesn't have the closure argument or the DispatchQueue argument.

Thoughts?

As noted in a pair of TODO comments in ExceptionTestingTests.swift, we do not currently have way of testing that functions should mark a test as failing actual do that without failing the MTK test itself.

Ever since the end of May this year, CocoaDocs no longer generates and hosts documentation for newly submitted pod versions. Read more here.

We need to figure out a simple way of generating and hosting documentation for new versions ourselves. Generating the documentation is fairly simple--we can just use jazzy, which is the tool CocoaDocs used. The trickier parts that we need to address are:

• Automate documentation generation each time we submit a new version.
• Hosting the documentation ourselves.

I've looked into it a little bit so far, and I think we could potentially simply host our docs within our GitHub repo to make publishing new docs easy, but we'd want to automate this process. You can check out the branch I've started to see what I've done so far:

• Branch
• Docs Preview (Using rawgit.com)

So far, the main problems with what I've done on that branch are:

• We have to remember to run jazzy each time before submitting a new version.
• We have to remember to update the documentation link in the README so it references the index page of the latest docset.
• The documentation badge isn't working on the README. This may be a simple fix though.

@nhgrif I had to do this because it was driving me nuts.

https://www.cultofpedagogy.com/two-spaces-after-period/ http://www.quickanddirtytips.com/education/grammar/two-spaces-after-a-period http://www.slate.com/articles/technology/technology/2011/01/space_invaders.html

😬

We can now write unit tests to verify that our custom assertion functions actually fail tests.

This PR resolves https://github.com/metova/MetovaTestKit/issues/2

The primary motivation behind this PR is to update the way test failures are checked.

When adding the main target assertion for comparing dates, I ran into a problem where the order of the sub-parts of the failure message could not be guaranteed due to the use of a set. Yet the way we were checking for assertions was via a direct string comparison.

In order to deal with this, the TestFailureExpectation struct was turned into a protocol. This protocol defines three optional functions. They default to returning TestVerificationResult.expected, basically ignore that specific check.

A BasicTestFailureExpectation class was also added in order to retain the exist test use-cases.

Ultimately though, this change allows the addition of the DetailedTestFailureExpectation (which may need a better name?), which allows to simply check that the description of the failure contains certain substrings (in any particular order).

This then allowed for the update of the date tests to assert that all of the mismatched components are reported in the case of more than one mismatched component, which is what we want (and could not previously test consistently, due to non-guaranteed order of items in a set).

This protocol-oriented approach to the test failure expectations should be relatively future-proofed against unforeseen problems like what was run into while testing the date assertions.

I also updated hardcoded line numbers in several spots to be simply offsets from the current #line, which should decrease the likelihood that these tests just start failing (despite an easy fix even if they did) simply due to someone adding or subtracting some code further up the file.

MetovaTestKit already contains two assertions for dealing with Objective-C level exceptions, as documented here.

Should we include assertions for dealing with Swift level errors? I understand that both XCTAssertThrowsError and XCTAssertNoThrow exist, and I generally prefer to not rewrite Apple code, however I have two issues with both of these methods.

First, the signature for the Xcode Test functions is arguably confusing or misleading. We generally prefer closure arguments to be last so we can leverage Swift's nice trailing closure syntax, as we did with Exception Testing Functions. The Xcode Test default functions have the closure to be tested as the first argument. This is done presumably because all of the following arguments are optional (and only used in very rare occasion... they're for people doing things like what we're doing with MTK) and to be consistent with every other function in the Xcode Test suite, which is fair enough.

Second, and arguably more importantly, the Xcode Test functions do not allow you to have much information about the error that was thrown (other than what was printed), nor does it allow you to make further assertions about the thrown error. The first problem is relatively minor, but the second problem can potentially get in the way of writing a complete test suite, particularly in the case where you want to assert that an error was thrown. You certainly want to be sure the error was of the type you were expecting and it contained all the information you expected to, otherwise you might have false positive results. If we were to implement these assertions in MTK, we would return any thrown error we encountered, just as we do with Objective-C exceptions we catch in the exception testing methods.

However, I believe everything else in MTK is purely supplemental functions. These two additions would mark the first replacements for existing XCT functionality.

As this library grows, trying to document everything in the top level README file will become overbearing. The top-level README should be trimmed down to the following major sections: Requirements, Installation, Usage, Credits, License.

The Usage section itself can contain a link to the full set of documentation generated based on doc comments. It should also have a very limited set of actual code snippets demonstrating the usage of the library. For this, we should determine what are the most popular or appealing aspects of the library and document these there. This section should contain some prose about who this library is meant for, what you can accomplish with it, etc. Then, this section should also link out to more complete & detailed documentation, including multiple example code snippets for each chunk of the total library. Consider Alamofire's Documentation directory as a guideline toward the direction we should be looking at. However, I imagine each chunk of MetovaTestKit having its own markdown file with documentation. The top-level README can then link straight to each of these individual files.

I'd like to create some kind of tool that can verify that all the test files are added to the test target. It's not uncommon for a test file to get removed from a test target during a .xcodeproj merge conflict resolution. If that happens, entire test classes can ended up being abandoned and forgotten about for a long time.

We'll need to spend some time designing how we'd want this to work, but I'm thinking that no matter how we end up designing it, it's going to have to be some kind of command line tool that can be run as a run script build phase.

Although iOS seems to compare image assets correctly if they are loaded using the imageNamed function on UIImage, this only works because iOS uses the same image in memory for those objects. If the same image is loaded from another source (from the file system, using SDWebImage, etc.), this comparison will not work. To avoid this confusing scenario, an assertion should be added that compares the image data so we can know that two images are identical.

Currently, when an MTK assertion fails, the message Xcode generates always includes "XCTAssertTrue failed - " plus the message or our default message. If possible, we'd like this message to instead use the name of function out of MTK.