CloudCore is a framework that manages syncing between iCloud (CloudKit) and Core Data written on native Swift.

Overview

CloudCore

Platform Status Swift

CloudCore is a framework that manages syncing between iCloud (CloudKit) and Core Data written on native Swift.

Features

  • Leveraging NSPersistentHistory, local changes are pushed to CloudKit when online
  • Pull manually or on CloudKit remote notifications.
  • Differential sync, only changed object and values are uploaded and downloaded.
  • Core Data relationships are preserved
  • private database and shared database push and pull is supported.
  • public database push is supported
  • Parent-Child relationships can be defined for CloudKit Sharing
  • Respects Core Data options (cascade deletions, external storage).
  • Knows and manages CloudKit errors like userDeletedZone, zoneNotFound, changeTokenExpired, isMore.

CloudCore vs iOS 13?

At WWDC 2019, Apple announced support for NSPersistentCloudKitContainer in iOS 13, which provides native support for Core Data <-> CloudKit synchronization. Here are some initial thoughts on the differences between these two approaches.

NSPersistentCloudKitContainer
  • Simple to enable
  • Private Database only, no Sharing or Public support
  • Synchronizes All Records
  • No CloudKit Metadata (e.g. recordName, systemFields, owner)
  • Record-level Synchronization (entire objects are pushed)
  • Offline Synchronization is opaque, but doesn't appear to require NSPersistentHistoryTracking
  • All Core Data names are preceeded with "CD_" in CloudKit
  • Core Data Relationships are mapped thru CDMR records in CloudKit
  • Uses a specific custom zone in the Private Database
CloudCore
  • Support requires specific configuration in the Core Data Model
  • Support for Private, Shared, and Public databases
  • Selective Synchronization (e.g. can delete local objects without deleting remote records)
  • Explicit CloudKit Metadata
  • Field-level Synchronization (only changed attributes are pushed)
  • Offline Synchronziation via NSPersistentHistoryTracking
  • Core Data names are mapped exactly in CloudKit
  • Core Data Relationships are mapped to CloudKit CKReferences

During their WWDC presentation, Apple very clearly stated that NSPersistentCloudKitContainer is a foundation for future support of more advanced features #YMMV

How it works?

CloudCore is built using a "black box" architecture, so it works invisibly for your application. You just need to add several lines to your AppDelegate to enable it, as well as identify various aspects of your Core Data Model schema. Synchronization and error resolving is managed automatically.

  1. CloudCore stores change tokens from CloudKit, so only changed data is downloaded.
  2. When CloudCore is enabled (CloudCore.enable) it pulls changed data from CloudKit and subscribes to CloudKit push notifications about new changes.
  3. When CloudCore.pull is called manually or by push notification, CloudCore pulls and saves changed data to Core Data.
  4. When data is written to your persistent container (parent context is saved) CloudCore finds locally changed data and pushes to CloudKit.
  5. By leveraging NSPersistentHistory, changes can be queued when offline and pushed when online.

Installation

CocoaPods

CloudCore is available through CocoaPods. To install it, simply add the following line to your Podfile:

pod 'CloudCore'

How to help?

Current version of framework hasn't been deeply tested and may contain errors. If you can test framework, I will be very glad. If you found an error, please post an issue.

Quick start

  1. Enable CloudKit capability for you application: CloudKit capability

  2. For each entity type you want to sync, add this key: value pair to the UserInfo record of the entity:

  • CloudCoreScopes: private
  1. Also add 4 attributes to each entity:
  • privateRecordData attribute with Binary type
  • publicRecordData attribute with Binary type
  • recordName attribute with String type
  • ownerName attribute with String type
  1. And enable 'Preserve After Deletion' for the following attributes
  • privateRecordData
  • publicRecordData
  1. Make changes in your AppDelegate.swift file:
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {
  // Register for push notifications about changes
  application.registerForRemoteNotifications()

  // Enable CloudCore syncing
  CloudCore.enable(persistentContainer: persistentContainer)

  return true
}

// Notification from CloudKit about changes in remote database
func application(_ application: UIApplication, didReceiveRemoteNotification userInfo: [AnyHashable : Any], fetchCompletionHandler completionHandler: @escaping (UIBackgroundFetchResult) -> Void) {
  // Check if it CloudKit's and CloudCore notification
  if CloudCore.isCloudCoreNotification(withUserInfo: userInfo) {
    // Fetch changed data from iCloud
    CloudCore.pull(using: userInfo, to: persistentContainer, error: nil, completion: { (fetchResult) in
      completionHandler(fetchResult.uiBackgroundFetchResult)
    })
  }
}
  1. If you want to enable offline support, enable NSPersistentHistoryTracking when you initialize your Core Data stack
lazy var persistentContainer: NSPersistentContainer = {
	let container = NSPersistentContainer(name: "YourApp")

	let storeDescription = container.persistentStoreDescriptions.first
	storeDescription?.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey)

	container.loadPersistentStores { storeDescription, error in
		if let error = error as NSError? {
			// Replace this implementation with code to handle the error appropriately.                
		}
	}
	return container
}()
  1. To identify changes from your app that should be pushed, save from a background ManagedObjectContexts named CloudCorePushContext
persistentContainer.performBackgroundTask { moc in
	moc.name = CloudCore.config.pushContextName
	// make changes to objects, properties, and relationships you want pushed via CloudCore
	try? context.save()
}
  1. Make first run of your application in a development environment, fill an example data in Core Data and wait until sync completes. CloudKit will create needed schemas automatically.

Service attributes

CloudCore stores CloudKit information inside your managed objects, so you need to add attributes to your Core Data model for that. If required attributes are not found in an entity, that entity won't be synced.

Required attributes for each synced entity:

  1. Private Record Data attribute with Binary type
  2. Public Record Data attribute with Binary type
  3. Record Name attribute with String type
  4. Owner Name attribute with String type

You may specify attributes' names in one of two 2 ways (you may combine that ways in different entities).

Default names

The most simple way is to name attributes with default names because you don't need to map them in UserInfo.

Mapping via UserInfo

You can map your own attributes to the required service attributes. For each attribute you want to map, add an item to the attribute's UserInfo, using the key CloudCoreType and following values:

  • Private Record Data value is privateRecordData.
  • Public Record Data value is publicRecordData.
  • Record Name value is recordName.
  • Owner Name value is ownerName.

Model editor User Info

đź’ˇ Tips

  • I recommend to set the Record Name attribute as Indexed, to speed up updates in big databases.
  • Record Data attributes are used to store archived version of CKRecord with system fields only (like timestamps, tokens), so don't worry about size, no real data will be stored here.

CloudKit Sharing

To enable CloudKit Sharing when your entities have relationships, CloudCore will look for the following key:value pair in the UserInfo of your entities:

CloudCoreParent: name of the to-one relationship property in your entity

Example application

You can find example application at Example directory.

How to run it:

  1. Set Bundle Identifier.
  2. Check that embedded binaries has a correct path (you can remove and add again CloudCore.framework).
  3. If you're using simulator, login at iCloud on it.

How to use it:

  • + button adds new object to local storage (that will be automatically synced to Cloud)
  • refresh button calls pull to fetch data from Cloud. That is useful button for simulators because Simulator unable to receive push notifications
  • Use CloudKit dashboard to make changes and see it at application, and make change in application and see ones in dashboard. Don't forget to refresh dashboard's page because it doesn't update data on-the-fly.

Tests

CloudKit objects can't be mocked up, that's why I create 2 different types of tests:

  • Tests/Unit here I placed tests that can be performed without CloudKit connection. That tests are executed when you submit a Pull Request.

  • Tests/CloudKit here located "manual" tests, they are most important tests that can be run only in configured environment because they work with CloudKit and your Apple ID.

    Nothing will be wrong with your account, tests use only private CKDatabase for application.

    Please run these tests before opening pull requests. To run them you need to:

    1. Change TestableApp bundle id.
    2. Run in simulator or real device TestableApp target.
    3. Configure iCloud on that device: Settings.app → iCloud → Login.
    4. Run CloudKitTests, they are attached to TestableApp, so CloudKit connection will work.

Roadmap

  • Move beta to release status
  • Add CloudCore.disable method
  • Add methods to clear local cache and remote database
  • Add error resolving for limitExceeded error (split saves by relationships).

Authors

deeje cooley, deeje.com

  • added NSPersistentHistory and CloudKit Sharing Support

Vasily Ulianov, [email protected] Open for hire / relocation.

  • implemented version 1 and 2, with dynamic mapping between CoreData and CloudKit

Oleg MĂĽller

  • added full support for CoreData relationships
Comments
  • Deleted object not saving to iCloud

    Deleted object not saving to iCloud

    I've tried and tried but have not had any luck saving deleted object to iCloud. Everything else works fine. I have looked at the example repeatedly with no luck. I am trying to delete as shown below:

    func tableView(_ tableView: UITableView, commit editingStyle: UITableViewCell.EditingStyle,    forRowAt indexPath: IndexPath) {
        let anObject = fetchedResultsController.object(at: indexPath)
        let objectID = anObject.objectID
        if editingStyle == .delete {
            persistentContainer.performBackgroundTask { (moc) in
                moc.name = CloudCore.config.pushContextName
                if let objectToDelete = try? moc.existingObject(with: objectID) {
                    moc.delete(objectToDelete)
                    try? moc.save()
                }
            }
            print("In editingStyle == .delete.")
        }//End if editingStyle
    } //End tableview editingStyle
    

    I have also tried:

    func tableView(_ tableView: UITableView, commit editingStyle: UITableViewCell.EditingStyle, forRowAt indexPath: IndexPath) {
        if editingStyle == .delete {
            // Fetch Bullets
            let bullets = fetchedResultsController.object(at: indexPath)
            // Delete Bullets
            bullets.managedObjectContext?.delete(bullets)
            print("In editingStyle == .delete.")
        }//End if editingStyle
    } //End tableview editingStyle
    

    And save here:

    func controllerDidChangeContent(_ controller: NSFetchedResultsController<NSFetchRequestResult>) {
        tableView.endUpdates()
        persistentContainer.performBackgroundTask { moc in
            moc.name = CloudCore.config.pushContextName
            try? moc.save()
            print("Saving to CoreData from controllerDidChangeContent...")
        }
        tableView.reloadData()
        updateView()
    }//End controllerDidChangeContent
    

    Please help. I'm not sure what I am doing wrong.

    opened by noecantu 6
  • Ignore syncing individual attributes?

    Ignore syncing individual attributes?

    Is there some way I could mark certain attributes to not sync to CloudKit? I have an "order" field for sorting items, but want each device to be able to sort on its own. Any advice would be appreciated, thanks!

    opened by dippnerd 3
  • Switching both organization and employee to public in example fails on employee

    Switching both organization and employee to public in example fails on employee

    Unfortunately everything was working great in the example when using private scope, but once I switched to public scope, resetting the development environment first, the employee no longer saves and errors are displayed. I'm happing to dig in on some debugging if you can point me in the right direction:

    image image image image image image image

    Clicking on the record for an employee in the CloudKit Dashboard yields the following error: image

    opened by Cyclic 2
  • Add semaphore to limit one change being processed at once

    Add semaphore to limit one change being processed at once

    When we create multiple objects in quick succession there is a race condition where the same records are added to by synced multiple times, this causes a CKError and stops the rest of the batch from completing.

    Adding a semaphore here ensures that only one of the processChanges() blocks in run at once and therefore ensures that records are converted, processed and removed from history all before running again ensuring no duplicates.

    opened by davidrothera 2
  • need cache management for Data attributes, backed by CKAsset

    need cache management for Data attributes, backed by CKAsset

    CKAssets can be very large, and client apps will want to manage the cache state for these. Would be great if CloudCore supported this natively, probably thru a specific set of attribute names in the CoreData schema, which clients can use to signal to CloudCore when (and where?) to download CKAssets.

    opened by deeje 1
  • Strange race condition

    Strange race condition

    I'm evaluating using CloudCore for a new project and was playing around with the example app and am running into a strange issue.

    If I insert a sample one at a time and wait for the sync to CloudKit to complete there is no issue however if I rapidly click 4 or more times (3 or less is fine) then errors are presented about a partial sync failure.

    The objects that the error is referring to have already been synced and look complete however any other objects which were not yet synced are left in an inconsistent state.

    Once the app is restarted the remaining objects are synced bringing the databases in perfect sync.

    If you disconnect the device/simulator from having an internet connection and insert the objects and then re-enable connectivity then things also sync fine so it only seems to happen when there is 4+ objects created and being synced at the same time.

    opened by davidrothera 1
  • Update to Swift 5?

    Update to Swift 5?

    I updated to Swift 5 and observed the following. I was able to silence 2-5 but am stuck on issue 1.

    1. ObjectToRecordConverter.swift shows the following: .../CloudCore/Pods/CloudCore/Source/Classes/Push/ObjectToRecord/ObjectToRecordConverter.swift:111:21: Initializer for conditional binding must have Optional type, not 'CKRecord'

    2. CloudSaveController.swift shows the following: .../CloudCore/Pods/CloudCore/Source/Classes/Push/PushOperationQueue.swift:75:50: Value of optional type 'URL?' must be unwrapped to a value of type 'URL'

    3. CloudCore.swift shows the following: .../CloudCore/Pods/CloudCore/Source/Classes/CloudCore.swift:139:48: Value of optional type 'CKNotification?' must be unwrapped to a value of type 'CKNotification'

    4. CloudCore.swift shows the following: .../CloudCore/Pods/CloudCore/Source/Classes/CloudCore.swift:180:25: Value of optional type 'CKNotification?' must be unwrapped to a value of type 'CKNotification'

    5. CloudKitAttribute.swift shows the following: .../CloudCore/Pods/CloudCore/Source/Model/CloudKitAttribute.swift:44:64: Value of optional type 'URL?' must be unwrapped to a value of type 'URL'

    opened by noecantu 1
  • Feature/Cacheable

    Feature/Cacheable

    Makes it easy for developers to implement schemas involving large data files, such as HD photos and 4K videos, by managing Upload and Download long-lived operations. These caching operations are automatically restored on app launch, and periodically save status and progress info which UI can observe and display.

    opened by deeje 0
  • Feature/4.0

    Feature/4.0

    Features

    • Sharing and SharingUI support
    • Updated Example app includes Sharing
    • add convenience func NSPersistentContainer.performBackgroundPushTask(…)
    • Add ability to pull a root record #3d9ffdfeb70e, #66d68ca9
    • Gracefully handle unknown references #c7e9a056

    Fixes

    • Broken sync tokens #0392745a
    • Workaround for bug in CloudKit zone name #5f5676be
    • Potential race condition in processChanges #32cdd72a
    • Improper access outside of context threads #e9181165
    • Serialize pull requests, to eliminate chance of duplicates #52f131a
    opened by deeje 0
  • Added enable(... withPull: Bool)

    Added enable(... withPull: Bool)

    Added enable(persistentContainer container: NSPersistentContainer, withPull: Bool) to allow developer to specify if Pull should be performed upon enable.

    opened by jfhamel 0
  • Reconsider publishing to

    Reconsider publishing to "public"

    I’m not necessarily thrilled with where support for “public” data landed. It was somewhat-implemented when I forked, and I reworked it for a scenario where an app could simultaneously publish certain record types to both private and public databases.

    I’d like to add support for an “isPublic” attribute that CloudCore could automatically process.

    opened by deeje 0
  • Creating data in public database does not sync with other devices

    Creating data in public database does not sync with other devices

    So far, I appear to be creating records successfully using the example app. image

    The first record was created using the simulator, and the second record created using my own physical device. However, it doesn't appear that the two instances of the app receive notifications for sync and so do not appear to be syncing at all. This is the boilerplate example app, which I've modified the entities to use public rather than private, and removed the organization parent as I've mentioned in a previous post.

    Do you have any advice on what might be causing the two devices not to sync? In addition, I had a separate user in a separate geographic location use the same container. He is able to create records in the same public CloudKit container, but again, we do not see updates from each other's app instances.

    Thanks in advance!

    Device A: image

    Device B: IMG_AECFC89149B3-1

    opened by Cyclic 6
  • Managed Object doesn't have stored record information

    Managed Object doesn't have stored record information

    Hi, execution stops at this line:

    		// That is possible if method is called before all managed object were filled with recordData
    		// That may cause possible reference corruption (Core Data -> iCloud), but it is not critical
    		assertionFailure("Managed Object doesn't have stored record information, should be reported as a framework bug")
    

    in file CoreDataRelationship.swift.

    It started occurring when I started using a NSManagedObject that inherits from another parent NSManagedObject. Never occurred before.

    Thanks.

    opened by jfhamel 0
  • Added enable(... withPull: Bool)

    Added enable(... withPull: Bool)

    Added enable(persistentContainer container: NSPersistentContainer, withPull: Bool) to allow developer to specify if Pull should be performed upon enable.

    Also, added init() {} to CloudCoreConfig to allow creation of instance.

    opened by jfhamel 0
  • Relationships between public and private data?

    Relationships between public and private data?

    There are many data scenarios which might involved public records and private records pointing to each other. How can CloudCore best support such scenarios?

    opened by deeje 0
  • Override Persistent History Clearing

    Override Persistent History Clearing

    currently, CloudCore assumes ownership of NSPersistentHistoryTracking, clearing out history after its been properly pushed to CloudKit. But NSPersistentHistoryTracking is also useful for shared containers and app extensions. Need to refactor such that apps can override default history clearing behavior.

    opened by deeje 1
Releases(5.1.0)
  • 5.1.0(Jun 1, 2022)

  • 5.0.0(May 15, 2022)

    Maskable Attributes enables you to identify which fields are to be excluded during fetch and modify operations between CloudKit and Core Data.

    Cacheable Assets builds on Maskable Attributes to ignore asset-type fields during fetch and modify operations, and enable you to configure your schema to support very large assets which can be uploaded and downloaded using Long-Lived CloudKit operations.

    Source code(tar.gz)
    Source code(zip)
  • 4.1.1(Apr 13, 2022)

  • 4.1(Apr 12, 2022)

  • 4.0.2(Apr 3, 2022)

  • 4.0.1(Apr 3, 2022)

  • 4.0(Aug 4, 2021)

    Features Sharing and SharingUI support Updated Example app includes Sharing add convenience func NSPersistentContainer.performBackgroundPushTask(…) Add ability to pull a root record #3d9ffdfeb70e, #66d68ca9 Gracefully handle unknown references #c7e9a056

    Fixes Broken sync tokens #0392745a Workaround for bug in CloudKit zone name #5f5676be Potential race condition in processChanges #32cdd72a Improper access outside of context threads #e9181165 Serialize pull requests, to eliminate chance of duplicates #52f131a

    Source code(tar.gz)
    Source code(zip)
  • 3.0.0(Aug 20, 2019)

    • Support for offline synchronization via NSPersistentHistory
    • Support for CoreData Relationships
    • Support for Shared and Public Databases
    • Swift 5.0
    Source code(tar.gz)
    Source code(zip)
Owner
deeje cooley
I create elegant mobile-cloud experiences.
deeje cooley
JSON to Core Data and back. Swift Core Data Sync.

Notice: Sync was supported from it's creation back in 2014 until March 2021 Moving forward I won't be able to support this project since I'm no longer

Nes 2.5k Dec 31, 2022
Robust CloudKit synchronization: offline editing, relationships, shared and public databases, field-level deltas, and more.

CloudCore CloudCore is a framework that manages syncing between iCloud (CloudKit) and Core Data written on native Swift. Features Leveraging NSPersist

deeje cooley 123 Dec 31, 2022
A powerful and elegant Core Data framework for Swift.

A powerful and elegant Core Data framework for Swift. Usage Beta version. New docs soon... Simple do that: let query = persistentContainer.viewContext

null 782 Nov 6, 2022
A synchronization framework for Core Data.

Core Data Ensembles Author: Drew McCormack Created: 29th September, 2013 Last Updated: 15th February, 2017 Ensembles 2 is now available for purchase a

Drew McCormack 1.6k Dec 6, 2022
Unleashing the real power of Core Data with the elegance and safety of Swift

Unleashing the real power of Core Data with the elegance and safety of Swift Dependency managers Contact Swift 5.4: iOS 11+ / macOS 10.13+ / watchOS 4

John Estropia 3.7k Jan 9, 2023
JustPersist is the easiest and safest way to do persistence on iOS with Core Data support out of the box.

JustPersist JustPersist is the easiest and safest way to do persistence on iOS with Core Data support out of the box. It also allows you to migrate to

Just Eat 167 Mar 13, 2022
A minimalistic, thread safe, non-boilerplate and super easy to use version of Active Record on Core Data.

Skopelos A minimalistic, thread-safe, non-boilerplate and super easy to use version of Active Record on Core Data. Simply all you need for doing Core

Alberto De Bortoli 235 Sep 9, 2022
Super awesome Swift minion for Core Data (iOS, macOS, tvOS)

⚠️ Since this repository is going to be archived soon, I suggest migrating to NSPersistentContainer instead (available since iOS 10). For other conven

Marko Tadić 306 Sep 23, 2022
HitList is a Swift App shows the implementation of Core Data.

HitList HitList is a Swift App shows the implementation of Core Data. It is the demo app of Ray Wenderlich's tech blog. For details please reference G

Kushal Shingote 2 Dec 9, 2022
A type-safe, fluent Swift library for working with Core Data

Core Data Query Interface (CDQI) is a type-safe, fluent, intuitive library for working with Core Data in Swift. CDQI tremendously reduces the amount o

null 31 Oct 26, 2022
100% Swift Simple Boilerplate Free Core Data Stack. NSPersistentContainer

DATAStack helps you to alleviate the Core Data boilerplate. Now you can go to your AppDelegate remove all the Core Data related code and replace it wi

Nes 216 Jan 3, 2023
This project server as a demo for anyone who wishes to learn Core Data in Swift.

CoreDataDemo This project server as a demo for anyone who wishes to learn Core Data in Swift. The purpose of this project is to help someone new to Co

null 1 May 3, 2022
A type-safe, fluent Swift library for working with Core Data

Core Data Query Interface (CDQI) is a type-safe, fluent, intuitive library for working with Core Data in Swift. CDQI tremendously reduces the amount o

null 31 Oct 26, 2022
JSQCoreDataKit - A swifter Core Data stack

JSQCoreDataKit A swifter Core Data stack About This library aims to do the following: Encode Core Data best practices, so you don't have to think "is

Jesse Squires 596 Dec 3, 2022
QueryKit, a simple type-safe Core Data query language.

QueryKit QueryKit, a simple type-safe Core Data query language. Usage QuerySet<Person>(context, "Person")

QueryKit 1.5k Dec 20, 2022
Core Data code generation

mogenerator Visit the project's pretty homepage. Here's mogenerator's elevator pitch: mogenerator is a command-line tool that, given an .xcdatamodel f

Wolf Rentzsch 3k Dec 30, 2022
Super Awesome Easy Fetching for Core Data!

MagicalRecord In software engineering, the active record pattern is a design pattern found in software that stores its data in relational databases. I

Magical Panda Software 10.9k Dec 29, 2022
A feature-light wrapper around Core Data that simplifies common database operations.

Introduction Core Data Dandy is a feature-light wrapper around Core Data that simplifies common database operations. Feature summary Initializes and m

Fuzz Productions 33 May 11, 2022
The Big Nerd Ranch Core Data Stack

BNR Core Data Stack The BNR Core Data Stack is a small Swift framework that makes it both easier and safer to use Core Data. A better fetched results

Big Nerd Ranch 561 Jul 29, 2022