Given an observable integer a, we can define the observable sum of adding it to itself:

import GlueKit
let a = Variable<Int>(2)

let sum = a + a

The value of sum seems to track the original observable correctly:

sum.value      // => 4
a.value = 1
sum.value      // => 2

However, if we look into the changes reported by sum, we find something surprising: apparently the result of adding an integer to itself can temporarily look like an odd number?!

var reported: [String] = []
let connection = sum.changes.connect { change in reported.append("\(change.old)→\(change.new)") }
a.value = 2
reported.joined(separator: ", ")  // => "2→3, 3→4"
connection.disconnect()

The reason for this is quite simple: as a changes from 1 to 2, the plus operator in a + a receives two changes, for both of its dependencies. It doesn't know these changes aren't independent, so it reports two separate changes to itself, going from 2 to 3, then 3 to 4.

These spurious notifications can cause issues when we build complex observable expressions. At the very least, they mean more changes are generated than strictly necessary, costing performance. But the real problem is that such temporary values aren't valid, and shouldn't be reported at all.

Hey, your library is really interesting.

The only problem I found was the README.md, which lacks an Installation Section I created this iOS Open source Readme Template so you can take a look on how to easily create an Installation Section If you want, I can help you to organize the lib.

What are your thoughts? 😄

As noted in issue #5, two-way bindings have not yet been updated to support transactional updates.

https://github.com/lorentey/GlueKit/blob/a0ed7f17fe0c05380270554c4ef79151f4cb473e/Sources/UpdatableValue.swift#L151-L171

Update the code to take transactions into account. This is more difficult than it seems! E.g., when b is bound to a, a beginTransaction update from a should immediately trigger the same from b. However, b's resulting beginTransaction message must not loop back to a, even though there is a two-way binding between them!

Mapping an array requires maintaining an index mapping, which is superfluous when we're going to ignore the ordering anyway; so array.map{$0.someInteger}.sum() does extra work.

Composite observables often need to observe many dependencies at once; e.g. array.flatMap { $0.observableField } needs to observe not only the array itself, but also the specified observable field of each of its individual elements. Currently, each individual observation requires at least five allocations:

• The subscription sink itself is always a closure that notifies the dependent observable; it is heap-allocated (allocation no. 1).
• The Connection disposable that controls the lifetime of the subscription is a heap-allocated object (allocation no. 2).
• Connection holds an array of disconnection callbacks and a lock:
  • The lock is of a class type (allocation no. 3).
  • The array's storage is heap allocated (allocation no. 4).
  • There is always at least one disconnection callback; it's a closure with a strong reference to the associated signal. It's heap-allocated (allocation no. 5).

This seems incredibly wasteful, and it should be changed to minimize the number of allocations. Ideally, we'd be able to subscribe to a set of observables with no per-subscription allocations.

Notes about fixing this

• SourceType should not be defined in terms of the type-lifted Sink struct; its connect function should be generic, so that we can use sinks that don't contain closures:

  public protocol SourceType {
     associatedtype SourceValue
     func connect<S: SinkType>(_ sink: S) -> Connection where S.SinkValue == SourceValue
  }
  struct FooSink {  // A sink that doesn't need anything allocated
    let target: Foo
    func receive(_ value: Int) { target.doSomething(value) }
  }
  

  (I think most aggregate sinks will fit the space reserved for such generic parameters without the allocation of a box.)

• We could convert Connection into a class hierarchy, and move the responsibility of storing the sinks into it. I.e., define

  class ConcreteConnection<Source: SourceType, Sink: SinkType>: BaseConnection, SinkType
  where Source.SourceValue == Sink.SinkValue {
    typealias Value = Source.SourceValue
    var source: Source
    var sink: Sink
  
    deinit { disconnect() }
    func disconnect() { source?.disconnect(self); source = nil; sink = nil }
    func receive(_ value: Value) { sink?.receive(value) }
  }
  

  Connection is currently a mish-mash of various hooks, bells and whistles (RefListElement, locking, additional callbacks, etc.). These should be ripped out and/or replaced by specific subclasses, as needed.

  In this approach, sources wouldn't create the connections any more — the connection would be created outside the source, then given to it using new API. (In this scheme, Connection should probably be renamed Sink, replacing the existing sink concept.)

• Ideally, Connection would be a protocol so that structs could also implement it. However, the protocol must not have associated types, so that it has existentials that can be stored in a collection. Therefore, it cannot contain the receive function above. However, the connection has a reference to the source and knows all types involved, so we could use it to define a workaround:

  public protocol SourceType {
     associatedtype SourceValue
     func register(_ connection: Connection)
     func unregister(_ connection: Connection)
     func _getValue() -> SourceValue // Only callable from Connection.receiveValue()
  }
  protocol Connection {
    func disconnect()
    func receiveValue() // Calls source._getValue() to get the value, and sends it to the sink
  }
  

  (Hopefully there is a way to make this less ugly.) This approach means that a single connection may not subscribe to more than one source at a time -- but that's not really a limitation, since instantiating a new collection gets much cheaper.

• A binary observable operator like + could then simply observe its dependencies without any allocations:

  struct LeftSink<Source: SourceType, Value: IntegerArithmetic>: SinkType, Connection 
  where Source.SourceValue == SimpleChange<Value> {
    let source: Source
    let target: BinaryCompositeObservable<Value>
    func receive(_ change: SimpleChange<Value>) { target.applyLeft(change) }
  }
  // struct RightSink is defined similarly.
  
  class BinaryCompositeObservable: ... {
    init<O: ObservableType>(left: O, right: O, combinator: @escaping (Value, Value) -> Value) {
       …
       left.changes.register(LeftSink(source: left.changes, target: self))
       right.changes.register(RightSink(source: left.changes, target: self))
    }
    deinit {
        left.unregister(LeftSink(source: left.changes, target: self))
        right.unregister(RightSink(source: left.changes, target: self))
    }
  }
  

• All of this should be done so that the closure-based API remains unchanged:

  let connection = signal.connect { value in print(value) }
  signal.send(2)
  signal.send(3)
  connection.disconnect()
  

  The compatibility definition of connect above would simply create a Connection/Sink containing the specified closure, register it to the source, and return it.

This PR massively reduces the cost of observations inside GlueKit's transformations by eliminating most heap allocations.

• From now on, types implementing SinkType need to be Hashable, providing an identity for subscribers. This gets rid of the need for a separate mechanism to assign identity to sinks, which simplifies Signal.
• SourceType.connect, which was previously the primary subscription interface, has been replaced by two methods: add and remove. Both of them take a single sink argument. remove returns the original sink instance; this is important when the sink has some hidden internal state on its own.
• The original closure-based connect interface is still provided as an extension method. It remains the preferred way to subscribe to observables outside of GlueKit.
• As a bonus change, UpdatableType has been improved by replacing withTransaction by an apply function that can take any update. This is far more generic and useful.

Remaining allocations:

1. Signals (and storage for the Set inside them) are lazily allocated when the first subscriber arrives at an observable. (Inlining Signals would eliminate one more allocation; but this is best done after #5 has been implemented — Signal implementation is much too complicated right now.)
2. SinkType is a protocol with an associated type, so Swift does not currently provide a protocol existential for it. GlueKit needs to implement type erasure using class polymorphism; so each sink needs to be wrapped inside a heap-allocated class instance. (I assume if generalized protocol existentials get implemented, they will include limited space for inline storage of small values, like Any does. Most GlueKit-internal sinks are of three words or less, which would hopefully fit inside the existential without boxing.)

This fixes issue #3.

Issue #6 updated bindings for the new transactional updates, but it did not add full support. Specifically, setting up or tearing down two-way bindings during an active transaction remains unsupported, and traps if you try.

It is unclear if this will cause problems in practical apps — assuming bindings are set up during in UIViewController's viewWillAppear and destroyed in viewDidDisappear, they would normally be outside of any active transaction.

If it did cause problems, however, we have are a number of ways available to support mid-transaction bindings.

1. Delay formally establishing/destroying the binding until the current transaction finishes. This seems easy enough to do (we just need to add a little more complicated state machine in BindConnection/BindSink), but it might break people's assumptions about how bindings interact with transactions. Note that changes must start getting propagated immediately when the binding is set up, and the value must stop getting synced immediately after unbinding—only transaction boundaries may get fuzzed when necessary. Bindings may survive BindConnection's deinit in this system, so state management must be moved to a new private class.

2. Handle the underlying issue of needing to keep track of the origin updatable for each transaction. (When unbinding the two updatables, we need to split the pool of active transactions by assigning each to the updatable that originated it.) One way to do this would be to change the transaction system such that not just the outermost begin/end messages are propagated to subscribers, but also nested ones. We would probably also need some form of unique transaction ID embedded in begin/end messages — the binding needs to recognize loopbacks for messages it forwarded itself and differentiate them from messages notifying about nested transactions coming from outside.

3. We could perhaps collapse updatables that are bound together into a single logical entity. The original components would still keep track of their subscribers and number of outstanding transactions, but they would let the unified entity handle all messaging. The problem with this approach is that it would considerably complicate the Source API.

Issue #5 disallows updates to an Observable from inside its own observers, but observers may still synchronously modify the state of other parts of the system.

This means that an observer to variable a is free to directly update the value of variable b, causing b's observers to be notified of the change immediately, even though a may not have finished notifying all of its observers of the original change. If an object was subscribed to both variables, it may thus receive b's notification before it sees the change to a that caused it — it may observe the effect before the cause!

Such retrocausality is against FRP's principles, but it may not actually cause serious problems. Find out if it needs to be fixed.

As an experiment, GlueKit's signal implementation currently provides support for reentrant sends; e.g., you are free to change the value of a variable from one of the subscribers to it:

let foo = Variable<Int>(4)
let c = foo.values.connect { v in 
    // If v is even, set it to a nearby odd value.
    if v & 1 == 0 { 
        foo.value = v + 1
    } 
}
print(foo.value)      // => 5
foo.value = 6
print(foo.value)      // => 7
c.disconnect()

Values that are sent while a signal is already sending values are queued up and sent asynchronously. Implementing this has been fun, but I wonder if it is worth the additional complexity.

Background

For reference, no reactive framework I know supports reentrant sends; they all deadlock on the first nested send invocation. However, KVO does support updating a property from one of its observers. When the property's value changes during one of KVO's signaling loops,

1. KVO performs a new, nested, synchronous signaling loop notifying all observers of the change immediately. The observer that was responsible for the nested change receives a nested observeValue(forKeyPath:,…) call.
2. Once the nested loop finishes, it completes the original signaling loop. However, it doesn't send the original notification payload, because that describes an outdated change — it updates the notification to match the latest value of the property.

KVO can do this because its signaling subsystem is written specifically for the purpose of sending change notifications, so it can update the notification payload as it is being sent, by merging interim changes into the original update.

GlueKit's Signal is a general signaling mechanism where in-flight values cannot be modified. So the only reasonable way to implement nested sends is to send their values asynchronously, after the currently running signaling loop completes. This means that the values that are received by observers are different in the two systems:

https://github.com/lorentey/GlueKit/blob/a0ed7f17fe0c05380270554c4ef79151f4cb473e/Tests/KVOSupportTests.swift#L217-L284

GlueKit's Signal is more consistent in the sense that everyone receives the same values, in the order they were sent. But KVO's implementation makes a little more sense in the context of change notifications — the value that observers see always matches what's in their notification payload.

Why is it a pain to support nested sends

The possibility that sending a value to an observer may trigger a nested send immensely complicates matters in a variety of common scenarios. For instance, sending a "welcome" value to new observers, like ObservableValueType.values does, becomes horribly complicated:

https://github.com/lorentey/GlueKit/blob/a0ed7f17fe0c05380270554c4ef79151f4cb473e/Sources/ObservableValue.swift#L62-L82

(The code is so complicated because it needs to handle the case where sink.receive(value) changes the value; for consistency, we want the sink to receive updates about its own changes, too. Also, connecting the sink to the source may send values to it by side effect (like values does), and those values need to be ordered consistently, too.)

I expect such nested sends will occur extremely rarely (if at all) in normal use-cases, but writing code to support them is unreasonably hard, and comes at a (probably) measurable cost of performance at runtime.

Furthermore, it is very easy to forget nested sends might occur; I'm sure the code already has instances where a nested send may lead to an inconsistent sequence of notifications.

How to remove support for reentrant sends

Signal can be easily simplified to remove support for nested sends; that's not a problem. Ideally a nested send would cause a trap, but deadlocking is fine, too, and is probably much easier to implement.

Other code that can be simplified can be found by a review of the codebase. (It's OK if we miss a couple.)

We need to consider whether KVO's support for reentrancy causes problems for the adapter code in KVO Support.swift. At first glance it doesn't, because recursive observeValue(forKeyPath:…) calls only happen for the actual observers that cause nested updates; other observers only get called strictly sequentially. (The fact that other reactive frameworks had no real issues with their own KVO adapters indicate this is correct.)

What about two-way bindings?

GlueKit includes rudimentary support for binding two updatables of the same Equatable type together.

https://github.com/lorentey/GlueKit/blob/a0ed7f17fe0c05380270554c4ef79151f4cb473e/Sources/UpdatableValue.swift#L151-L171

This feature has seen very limited use in production, and it is as yet unclear if the concept is robust enough. It has not even been updated yet for transactional changes. But supposing it is updated, removing support for nested sends does not cause problems at first glance — the equality check is there to prevent an infinite update loop, and it also does a good job of preventing even a single reentrant update. (Corner cases need to be dealt with, though. In the absence of nested updates, I don't think a Behavior's current value can ever differ from the one described in one its update Events, but I may be wrong.)