All of SkyLab's methods rely on blocks -- which is a great structure for control-flow between tests. Things look a little strange, though, once you want something synchronously. ie,

- (BOOL)showNewFancyButton 
{
     BOOL showNewFancyButton;
    [SkyLab abTestWithName:@"ShowNewFancyButton" A:^{
        showNewFancyButton = YES;
    } B:^{
        showNewFancyButton = NO;
    }]

    return showNewFancyButton;
}

Looking at the implementation of SkyLab, the above code will work just fine, but we thought it's a little weird to rely on blocks to run synchronously (even though we know the implementation does).

Do you think it'd be cool for SkyLabs to have non-block counterparts to its block-based methods? Or would you just suggest doing synchronous operations like above?

This warning is emanated from SkyLab.m line 67:

double r = (random() % (INT_MAX + 1)) * (total / INT_MAX);

I assume INT_MAX + 1 is supposed to be INT_MIN? Or is the overflow intentional?

Thank first. I am already using SkyLab and GroundControl.

It makes life easy to init a array or dictionary using @[] or @{}. But it will be a problem for those whose XCode is still under 4.2. I encountered this problem.

In the code you init a array using this:

@[ @"A", @"B" ]

would you please change it to [NSArray arrayWithObjects:@"A", @"B", nil]?

It would be nice to hear thoughts on how to integration with a backend. Not about how to do network requests, but things like should I just read keys out of NSUserDefaults, what should I do with them to make a meaningful conclusion?

I also didn't see an API for recording outcomes (positive/negative). How do you handle that?

Thanks

It looks like:

    if ([choices isKindOfClass:[NSArray class]]) {
        if (!choice || ![choices containsObject:choice]) {
            choice = SLRandomValueFromArray(choices);
        }
    } else if ([choices isKindOfClass:[NSDictionary class]]) {
        if (!choice || ![[choices allKeys] containsObject:choice]) {
            choice = SLRandomKeyFromDictionaryWithWeightedValues(choices);
        }
    } else {
        @throw [NSException exceptionWithName:NSInvalidArgumentException reason:NSLocalizedString(@"Parameter `choices` must be either array or dictionary", nil) userInfo:nil];
    }

    [[NSUserDefaults standardUserDefaults] setObject:choice forKey:SLUserDefaultsKeyForTestName(name)];
    [[NSUserDefaults standardUserDefaults] synchronize];

should change to:

    if ([choices isKindOfClass:[NSArray class]]) {
        if (!choice || ![choices containsObject:choice]) {
            choice = SLRandomValueFromArray(choices);
        }
        [[NSUserDefaults standardUserDefaults] setObject:choice forKey:SLUserDefaultsKeyForTestName(name)];
        [[NSUserDefaults standardUserDefaults] synchronize];
    } else if ([choices isKindOfClass:[NSDictionary class]]) {
        if (!choice || ![[choices allKeys] containsObject:choice]) {
            choice = SLRandomKeyFromDictionaryWithWeightedValues(choices);
        }
        [[NSUserDefaults standardUserDefaults] setObject:choice forKey:SLUserDefaultsKeyForTestName(name)];
        [[NSUserDefaults standardUserDefaults] synchronize];
    } else {
        @throw [NSException exceptionWithName:NSInvalidArgumentException reason:NSLocalizedString(@"Parameter `choices` must be either array or dictionary", nil) userInfo:nil];
    }

It looks like the same pattern is used in multivariateTestWithName:variables:block as well.

We ran across this because we were calling splitTestWithName:choices:block: from inside tableView:cellForRowAtIndexPath: and noticed repeated NSUserDefaultsDidChangeNotifications in the logs. Granted, that's not a great way to use it and we've refactored.

Please let me know if you'd rather have a pull request.