On Friday, September 15 2017 I presented a brief, casual talk to my team on the goals of Swift 5, including ABI Stability and I gave a high level summary of Chris Lattner’s Concurrency Manifesto. While I didn’t understand all of the low level details, I found that I ignited a strong interest in the topic and will continue to read more about it as the proposal progresses as well as alternative proposals that may be submitted.

Download a PDF of my slides here

Important Resources

Concurrency Manifesto by C Lattner

Kicking off Concurrency Discussions by Ted Kremeneck

Async/Await Proposal for Swift by Lattner

Prototype for Async/Await by Lattner

Old Swift Document on Concurrency

Swift Evolution Goals for Swift 5

Swift ABI Stability Manifesto

Goals for Swift 5

Primary Focus: ABI Stability

Big Picture

  1. Source Compatability i.e New Compilers able to compile code written in an older Swift version
  2. Binary Framework & Runtime compatibility i.e Distribution of Frameworks in a binary that works across multiple Swift versions. Requires two things to be achieved:
- Module Format Stability i.e stabilizing the module file, compiler’s representation of public framework interfaces, so it can do type checking, code generation when compiling.
- ABI Stability i.e enabling binary compatibility between applications and libraries compiled with different Swift versions.

What is ABI Stability?

At runtime Swift binaries interact with other libraries and components through an ABI (Application Binary Interfaces). It defines low level details like how to call a function, how data is represented in memory and where metadata is and how to access it.

Currently Swift is not ABI stable, so each binary (App), bundles its own version of the Swift Dynamic Library. Swift doesn’t live on the iOS Operating System, it lives within each App.

That means Sephora Color IQ is using Swift 3.1, so it bundles Swift 3.1’s Dynamic Library (containing the 3.1 ABI) inside. But the Gap app is using Swift 2.3, so it bundles Swift 2.3 and it’s 2.3 ABI.

If Swift becomes ABI Stable, Swift will live within the iOS Operating System and it’s ABI will be compatible with every version of Swift. i.e Sephora Color iQ is using Swift 5.0, but Gap is using Swift 4.3, and their both consuming Swift ABI embedded in the Operating System.

Why does ABI Stability matter?

  • Bundle size is reduced
  • Language changes smaller / Less frequent
  • Less Migration
  • Developers can create Pre-compiled Frameworks in Swift (currently frameworks are compiled when compiling your app) because no need to embed Swift

Drawbacks?

  • Limits changes to the Public Interfaces and Symbols
  • Restricts the ways Swift can grow and evolve

String Ergonomics

– Aim to complete more work outlined in the String Manifesto
– Language level support for Regular Expressions?
– Performance Improvements to the internal implementation of String

Standard Library Improvements

– Only making minor changes to improve Standard Library where needed

Foundation Improvements

  • Improvements Foundation that make using the Cocoa SDK with Swift more seamless

Syntactic Additions

  • Syntactic changes add complexity to the language
  • May be made but only if well motivated, under intense scrutiny

Groundwork/Discussion for new Concurrency Model

  • Finalizing the model is a non-goal for Swift 5
  • Key focus area is designing language affordances for creating and using asynchronous APIs and problems caused by callback-heavy code

Changes to the Swift Evolution Process for Swift 5

  • Unlike Swift 4, there will not be Stage 1, Stage 2 for accepting proposals
  • Proposals are welcome until March 1, 2018
  • To mitigate the risks of proposals distracting from ABI Stability, EVERY evolution proposal will need a working implementation with test cases before it can be considered for review.

How to Submit a Proposal

Step 1 – Write Proposal, submit via Pull Request to swift-evolution repository
Step 2 – Core Team provides feedback
Step 3 – If positive feedback, author must provide an implementation prior to proposal being formally reviewed.

Concurrency Manifesto

Introduction

  • Focuses on Task-Based Concurrency Abstractions (common in Event-Driven, Client-Server Applications i.e responding to UI events or requests)
  • Swift 1…4 has avoided Concurrency, relied on OS/Library level abstractions (GCD, pthreads, NSThread, etc) to manage tasks.
  • It’s already possible to use GCD..etc, so the Goal is to make the Concurrency experience far better than it is today
  • Improve the concurrency story with Swift’s values – Design, Maintenance, Safety, Scalability, Performance and Excellence

Why a First Class Concurrency Model?

Asynchronous APIs are difficult to work

  • Error Handling, Callbacks, when performing multiple operations creates complicated control flows
  • Made worse with Swift’s guard let syntax throughout callback closures

Hard to know what Queue/Thread you’re on

  • Swift closures don’t make it obvious which background thread or queue a task is being executed on
  • Leads to race conditions, and unexpected results

Shared mutable state Is bad for Software Developers

  • Data being mutated while someone else is reading from it
  • Typically solved using mutexes or locks, but this introduces a number of problems: ensuring you’re using the right locks, granularity of locks, avoid deadlocks and other issues.
  • Mutexes are inefficient
  • Techniques like Objective C read/copy/update are complex, unsafe and fragile

4 Major Abstractions in Computation

  • Traditional Control Flow
  • Asynchronous Control Flow
  • Message Passing and Data Isolation
  • Distributed Data and Compute

(1) Already exists in Swift
(2) Asynchrony is the next step towards Concurrency model. Fundamental to machines talking to other machines, slow devices and multiple operations.
(3) Next step is to define a programmer abstraction to define and model independent tasks in a program.

Current Asynchronous Solution in Swift

  • Passing “Completion handlers” using closures
  • Completion handlers stack up to a pyramid of doom
  • Make error handling awkward
  • Make control flow extremely difficult

Part 1 – Async/Await Pattern

Async

  • Well known solution used in other popular programming languages – C, C#, Python, Javascript, Scala with great success.
  • Async keyword used similar to the existing throws keyword
  • Declare a function as async to indicate function is a Coroutine.

Definition: Coroutines

  • Functions that may return a value normally, or may suspend themselves and internally return a continuation.

Before:

func loadWebResource(_ path: String, completionBlock: (result: Resource) -> Void) { ... }
func decodeImage(_ r1: Resource, _ r2: Resource, completionBlock: (result: Image) -> Void)
func dewarpAndCleanupImage(_ i : Image, completionBlock: (result: Image) -> Void)

func processImageData1(completionBlock: (result: Image) -> Void) {
loadWebResource("dataprofile.txt") { dataResource in
loadWebResource("imagedata.dat") { imageResource in
decodeImage(dataResource, imageResource) { imageTmp in
dewarpAndCleanupImage(imageTmp) { imageResult in
completionBlock(imageResult)
}
}
}
}
}

After:

func loadWebResource(_ path: String) async -> Resource
func decodeImage(_ r1: Resource, _ r2: Resource) async -> Image
func dewarpAndCleanupImage(_ i : Image) async -> Image

func processImageData1() async -> Image {
let dataResource = await loadWebResource("dataprofile.txt")
let imageResource = await loadWebResource("imagedata.dat")
let imageTmp = await decodeImage(dataResource, imageResource)
let imageResult = await dewarpAndCleanupImage(imageTmp)
return imageResult
}

Await

  • Similar to the existing try keyword.
  • Indicates that non-local control flow can happen at that point.

Part 2 – Actors

What is an Actor?

  • Actors represent real world concepts like “a document”, “a device”, “a network request”
  • Well suited to event driven architectures like UI applications and servers
  • An actor is a combination of a DispatchQueue, the data that queue protects and messages that can be run on the queue
  • An Actor would be a new ‘type’ in Swift, like class, struct or protocol
  • Allows programmer to define internal variables and functions to manage that data and perform operations on it
  • Actors can’t return values, throw errors or have inout parameters
actor TableModel {
let mainActor : TheMainActor
var theList : [String] = [] {
didSet {
mainActor.updateTableView(theList)
}
}

init(mainActor: TheMainActor) { self.mainActor = mainActor }

// this checks to see if all the entries in the list are capitalized:
// if so, it capitalize the string before returning it to encourage
// capitalization consistency in the list.
func prettify(_ x : String) -> String {
// Details omitted: it inspects theList, adjusting the
// string before returning it if necessary.
}

actor func add(entry: String) {
theList.append(prettify(entry))
}
}
  • UIKit and AppKit would model something like the ‘Main Thread’ using a ‘MainActor’
  • Programmers could define extensions to the MainActor in order to run their code on the main thread.
  • Actors are shutdown when their reference count reaches 0 and the last queued message is completed.

Part 3 – Fault Isolation

  • Far more complex and low level than Part 1 and Part 2
  • Most important thing to note is that by using the Actor model, state is no longer global
  • Therefore if a crash was to occur in a particular Actor, it could be possible to terminate just the Actor that had an issue instead of the whole process
  • This also has possible downsides such as if some other Actor or piece of code is awaiting that Actor to finish a task but it has terminated and will never return

Part 4 – System Architecture

  • Wayyyy more complex than everything else
  • Interprocess Communication and managing basic async operations is similar in many ways
  • Independent tasks communicating with each using by sending structured data using asynchronous messages
  • But there’s also a lot of things not similar
  • A Swift Developer shouldn’t have to worry about IPC if they don’t care about it
  • Actors can opt in to distribution using the distributed keyword
  • Requires that actors conform to a few different requirements in order to allow them to work with distributed systems.

Part 5 – The crazy and brilliant future

  • Room to extend upon Actors and asynchrony to improve long-standing Software Community divides (that are all way to complicated for me to mention here)

FIN