Written by Gaétan Zanella
on 

SwiftArgumentParser: A Swift mod

During the WWDC 2021, Apple promoted for the first time its new SwiftArgumentParser library. It aims to help creating command-line tools in Swift.

In just a few lines of code, we can build a fully documented command-line tool. Like in this example from its tiny (I’m pretty sure they made it tiny on purpose) README file:

struct Repeat: ParsableCommand {

    @Option(name: .shortAndLong, help: "The number of times to repeat 'phrase'.")
    var count: Int?

    @Argument(help: "The phrase to repeat.")
    var phrase: String

    mutating func run() throws {
        let repeatCount = count ?? .max
        for _ in 1...repeatCount {
          print(phrase)
        }
    }
}

Repeat.main()

We express our CLI as a graceful command tree while SwiftArgumentParser takes care of all the implementation details: it parses the command-line arguments, instantiates the target command, fulfills its arguments based on their names and finally runs it. It can even generate a built-in help description if a help option is passed:

$ repeat --help
USAGE: repeat [--count <count>] <phrase>

ARGUMENTS:
  <phrase>                The phrase to repeat.

OPTIONS:
  -c, --count <count>     The number of times to repeat 'phrase'.
  -h, --help              Show help for this command.

Apple brands its library as straightforward.

This achievement is big: even if you have never used the library, chances are you can already tell what the code does and how you could use it to create your own CLI command out of it.

Have you ever seen anything as straightfordward in Swift?

SwiftArgumentParser should remind you of SwiftUI. To create a new CLI option, you declare an @Option property without writing any parsing code. An approach that is similar to the declarative syntax of SwiftUI. In each case, thanks to this high level of abstraction, we only need to state what our code should do rather than writing it explicitly.

However, because Swift’s heavy focus on types, straightfordwardness is usually not the first quality that comes to mind. Contrary to dynamic languages, such as Ruby, Swift strongly relies on compilation-time validations, making it a safer, but significantly less flexible, language.

In fact, even after pondering on this for a bit, I have no idea how to implement such a behavior using Swift. SwiftArgumentParser is not a regular Swift library.

To corroborate, in the previous example, if instead of the main method, we used the autogenerated empty initializer of Repeat:

let command = Repeat()
command.run() // fatal error

This state-of-the-art, Swift library would just… crash! (In this case, Argument was initialized with only a basic help description, how could phrase return a string out of it?)

It seems Apple has to bypass some Swift compiler safety checks to offer such an API. It decided to do it even if it could obviously lead to serious codebase issues. Something we, thorough Swift developers, are not used to doing when targeting a production environment. Something we are not used to seeing from Apple.

Starting from this point, I knew the SwiftArgumentParser repository would contain some cryptic but powerful Swift usages. I had to deep dive inside it and figure out how Apple managed to make the magic happen.

Recreating the smallest SwiftArgumentParser API

There are a lot of blogposts out there on how to use SwiftArgumentParser. In this one, I will instead highlight the main implementation tricks Apple used to provide such beautiful API and what is the cost of it.

To that purpose, I developed a basic version of the library from scratch: FBSwiftArgumentParser.

The library only reproduces the following key features:

Similarly to the SwiftArgumentParser, an example of FBArgumentParser in a README file would look like this:

struct FBRepeat: FBParsableCommand {

    @FBOption(help: "The phrase to repeat.")
    var phrase = "Hello" // default value 😎

    mutating func run() throws {
        for i in 1...2 {
          print(phrase)
        }
    }
}

FBRepeat.main()

And its usage like that:

> fbrepeat --phrase "Bonjour"
Bonjour
Bonjour

> fbrepeat
Hello
Hello

> fbrepeat --help
fbrepeat
--phrase (optional) (default Hello) The phrase to repeat.

Let’s see how to implement it step by step while keeping it straightforward.

Defining the main FBParsableCommand protocol

Of course, our implementation starts with the library’s main protocol: FBParsableCommand.

public protocol FBParsableCommand {
  mutating func run() throws
}

It is a very basic version of the ParsableCommand. For example it does not support nesting commands nor contains those fancy properties to customize its help description. But that’s enough for now. As the original one, it will be used by the users of the library to declare their program logic.

We also define the same static main entry point in an extension (the users are not supposed to override it) as a static method:

public extension FBParsableCommand {

  static func main() {
    // TODO
  }
}

The execution steps of the main method are described in the SwiftArgumentParser’s README file:

You kick off execution by calling your type’s static main() method. The SwiftArgumentParser library parses the command-line arguments (#1), instantiates your command type (#2), and then either executes your run() method or exits with a useful message (#3).

Let’s write those steps.

First, we create a FBCommandParser object to encapsulate the command line arguments parsing code.

struct FBCommandParser {

    let command: FBParsableCommand.Type

    func parse(arguments: [String]) throws -> FBParsableCommand {
        let arguments = parseArguments(arguments) // # 1
        // TODO #2
    }

    // creates a dict from the command line arguments:
    // `command --arg1 "value1" --arg2 "value2"` => ["arg1": "value1", "arg2": "value2"]
    private func parseArguments(_ arguments: [String]) throws -> [String: String] {
        var argumentsToParse = arguments
        var result: [String: String] = [:]
        while argumentsToParse.count >= 2 {
            let argument = argumentsToParse.removeFirst()
            let isAnArgument = argument.starts(with: "-") || argument.starts(with: "--")
            if !isAnArgument {
                throw ParsingError.invalidParameters
            }
            result[argument.replacingOccurrences(of: "-", with: "")] = argumentsToParse.removeFirst()
        }
        return result
    }
}

As we only support CLI option type arguments (“–option value” or “-o value”), we store those key-value arguments in a basic dictionary. We would need something more sophisticated if more argument types were supported, but this will do for now.

Then, we execute the run method of the created instance and neatly exit the program if an error occured:

public extension FBParsableCommand {

    static func main() {
        do {
            let arguments = CommandLine.arguments.dropFirst() // we always ignore the first argument
            var command = try parseAsRoot(Array(arguments)) // #1 #2
            try command.run() // #3
        } catch {
            print(error.localizedDescription) // useful message in case of error
            exit(EXIT_FAILURE)
        }
    }

    private static func parseAsRoot(_ arguments: [String]) throws -> FBParsableCommand {
        let parser = FBCommandParser(command: self)
        return try parser.parse(arguments: arguments)
    }
}

We purposedly left a huge TODO in FBCommandParser. We basically still have to implement the core of the program: instantiating the target command and filling its properties based on the parsed CLI arguments dictionary.

Instantiating client commands

FBParsableCommand obviously needs an initiliazer constraint: main is a static method and run an instance one.

Adding a basic init(arguments: [String]) would work from an implementation perspective:

public protocol FBParsableCommand {
  init(arguments: [String: String])
}

But it would require a lot work for the library’s users to implement it. The user would have to manually fulfill its properties based on the given dictionary. Our library should do all the heavy lifting.

This is where SwiftArgumentParser shined for the first time. It used a very clever technic:

It uses Decodable to make the compiler generate all the argument parsing code on the client side in one fell swoop

In fact, if we consider the [String: String] argument array as a basic formatted data, we can consider FBParsableCommand as a custom data type. Thus if we can make the FBParsableCommand inherit from Decodable:

public protocol FBParsableCommand: Decoding {}

All the decoding code can be generated at compilation time right inside the command implementation!

Let’s see it in action:

We first define a basic custom Decoder object able to return the value associated to the given argument property.

struct FBOptionsDecoder: Decoder {

  let arguments: [String: String]

  // MARK - Public

  func value(for key: String?) -> String? {
    arguments[key]
  }

  // MARK - Decoder

  // ...

  func container<Key>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> where Key : CodingKey {
      let container = KeyeredArgumentContainer(decoder: self, type: type)
      return KeyedDecodingContainer(container)
  }
}

class KeyeredArgumentContainer<K: CodingKey>: KeyedDecodingContainerProtocol {

    let decoder: FBOptionsDecoder

    init(decoder: FBOptionsDecoder, type: K.Type) {
        self.decoder = decoder
    }

    // ...

    func decode<T>(_ type: T.Type, forKey key: K) throws -> T where T: Decodable {
        let value = decoder.value(for: key.stringValue)
        if let element = value as? T {
            return element
        } else {
          thow ParsingError.missingValue
        }
    }
}

It is almost straightforward as the value associated to a CLI option is already mapped to its name by our FBCommandParser.parseArguments method. We just need to use the decoding key as a search key in the parsed dictionary.

Then, we can use our custom Decoder to instanciate the target command, thanks to its conformance to Decodable:

struct FBCommandParser {

    enum ParsingError: Error {
        case invalidParameters
    }

    let command: FBParsableCommand.Type

    func parse(arguments: [String]) throws -> FBParsableCommand {
        let argumentDict = parseArguments(arguments)
        let decoder = FBOptionsDecoder(arguments: argumentDict)
        return try command.init(from: decoder)
    }

    // ...
}

Now, as long as the client command properties implement Decodable:

struct FBRepeat: FBParsableCommand {

    let phrase: String

    mutating func run() throws {
        for i in 1...2 {
          print(phrase)
        }
    }
}

FBRepeat.main()

The Swift compiler will automatically generate the appropriate Decoder method calls to fulfill all the properties of the target command.

> fbrepeat --phrase "Bonjour"
Bonjour
Bonjour

Notice though that FBRepeat does not manage default values correctly yet. If we set a default value to a property:

struct FBRepeat: FBParsableCommand {

    var phrase: String = "Hello"

    mutating func run() throws {
        for i in 1...2 {
          print(phrase)
        }
    }
}

FBRepeat.main()

It will be ignored by the generated implementation of the Decodable protocol:

> fbrepeat
Decoding ERROR

It seems like a small problem but it would force users to provide a custom implementation of Decodable in order to add default argument. We do not want that. We need to provide the users with an easy way to customize the decoding of its command.

You’ve guessed it, it is time to introduce property wrappers!

Using property wrappers as argument descriptions

In SwiftArgumentParser the information that we need to collect from the command line is defined using property wrappers. Each one of them matches to a type of argument (@Flag, @Option, @Argument etc) and provides initiliazers to describe them. We can specify how to parse the argument, its potential default value, its help description etc.

Let’s add a similar declarative API in FBSwiftArgumentParser and see how we can use it to tweak the parsing behavior.

We start with a basic FBOption:

@propertyWrapper
public struct FBOption {

    public var wrappedValue: String

    public init() {
        self.wrappedValue = ""
    }

    public init(wrappedValue: String) {
        self.wrappedValue = initialValue
    }
}

It has two public initializers so clients can declare two kinds of arguments:

Its conformance to Decodable is straightforward, thanks to singleValueContainer, so we keep the previous benefits provided by the Swift compiler:

extension FBOption: Decodable {

    public init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        wrappedValue = try container.decode(String.self))
    }
}

struct FBRepeat: FBParsableCommand {

    @FBOption
    var phrase: String = "Hello"

    mutating func run() throws { ... }
}

As expected, our code is still as broken as before though:

> fbrepeat --phrase "Bonjour"
Bonjour
Bonjour

> fbrepeat
Decoding ERROR

When no value is specified, FBCommand.main method still tries to decode the command using FBOption.init(from:) without being able to provide a proper value for the phrase argument. The declared default “Hello” value is ignored.

Let’s adapt our parsing code and take into account the potential default values provided by the property wrappers.

The two faces of the command property wrappers

This is probably the most confusing aspect of the implementation.

SwiftArgumentParser property wrappers have two usages depending on their initiliazation

Because of their declarative interfaces, when decoded, a property wrapper provides a value that can be used by the user during the execution of the command. Otherwise, it only provides a description of itself, used internally, that describes how to decode it.

Let’s highlight those two aspects in FBOption;

In code, we can represent the two cases using an enum:

struct FBOptionDefinition {
    let isOptional: Bool
    let defaultValue: String?
}

@propertyWrapper
public struct FBOption {

    enum State {
        case definition(FBOptionDefinition)
        case resolved(String)
    }

    private let state: State
}

In the first case, FBOption only wraps a FBOptionDefinition:

public extension FBOption  {

    public init(initialValue: String) {
        self.state = .definition(FBOptionDefinition(isOptional: true, defaultValue: initialValue))
    }

    public init() {
        self.state = .definition(FBOptionDefinition(isOptional: false, defaultValue: nil))
    }
}

In the other case, FBOption stores its actual value:

extension FBOption: Decodable {

    public init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        state = .resolved(try container.decode(String.self)))
    }
}

As a result, FBOption can only return a value if it is decoded:

public struct FBOption {

    public var wrappedValue: String {
        switch state {
            case .definition:
                fatalError("FBOption is not decoded")
            case let .resolved(value):
                return value
        }
    }
}

This is the issue we faced in the introduction and obviously a tradeoff: FBOption has two usages but only one non optional public API.

However, as long as the library clients rely on the main method to execute their program, we can ensure only the autogenerated Decodable.init(from:) initializer of the parsable command will be used to call the run method. Thus each property wrappers will be decoded and return a value when the client tries to read a potential CLI option.

Furthermore, when a user declares a command, it now implicitly declares a list of FBArgumentDefinition:

struct Repeat: FBParsableCommand {

    @FBOption
    var myUsualParameter1: String = "Hello" // uses `FBOption.init(wrappedValue:)` thus it wraps an `FBArgumentDefinition`

    @FBOption
    var myUsualParameter2: String // uses `FBOption.init()` thus it wraps an `FBArgumentDefinition`
}

Let’s see how we can use them in our decoding code to ensure the optional values provided will be used when the decoding occurs.

Using property wrappers as reflective markers

Even if it seems at odds with its heavy focus on compile-time validation, Swift enables us to inspect, and work with, the members of a type — dynamically, at runtime. It is by using this uncommon capacity of Swift that Apple implemented the decoding of the property wrappers.

SwiftArgumentParser uses metaprogramming

Let’s see how it works.

We first need a new constraint on the protocol:

public protocol FBParsableCommand: Decodable {
    init()
    mutating func run() throws
}

This way, we force the user to provide a way to instantiate a command in a proper definition state, using the definition given by each property’s wrapper.

In fact, we could be confident that the users will only use descriptive wrapper initializers in this case:

struct UsualRepeat: ParsableCommand {

    @FBOption()
    var usualProperty: Sting
}

Seeing the use of a decoding one, even if it compiles, would be pretty unusual:

let decoder: Decoder = ...

struct UnusualRepeat: ParsableCommand {

    @FBOption(from: decoder) // compiles, but unexpected, and will crash
    var unusualProperty: Sting
}

Thanks to this assumption, as long as the user respects the contract, by instanciating a blank version of the target command, we can use FBOption as a reflective marker and retrieve all its argument definitions:

private extension FBOption {

    func definition() -> FBOptionDefinition {
        switch state {
            case let .definition(definition):
                return definition
            case .resolved:
                fatalError("FBOption is already decoded")
        }
    }
}

private extension FBParsableCommand {

    // extracts all the argument definitions from the command type using mirroring
    static func argumentDefinitions() -> [String: FBOptionDefinition] {
        var definitionByKey: [String: FBOptionDefinition] = [:]
        let blankInstance = Self.init()
        Mirror(reflecting: blankInstance).children.forEach { child in
            guard let codingKey = child.label, let definition = (child.value as? FBOption)?.definition() else { return }
            // property wrappers are prefixed with "_"
            let sanitizedCodingKey = String(codingKey.first == "_" ? codingKey.dropFirst(1) : codingKey.dropFirst(0))
            definitionByKey[sanitizedCodingKey] = definition
        }
        return definitionByKey
    }
}

Notice that we need to parse the property wrapper names to match them with their corresponding key, each property wrapper starts with an underscore prefix.

We can now easily retrieve all the potential default values of a command with their corresponding arguments:

struct FBCommandParser {

    let command: FBParsableCommand.Type

    private func defaultParameters() -> [String: String] {
        command.argumentDefinitions().compactMapValues { $0.defaultValue }
    }
}

By merging them with the initial CLI arguments dictionary, we can thus easily fill the potential gaps when initializing the target command, using this time its Decoding.init(from:) initializer:

struct FBCommandParser {

    let command: FBParsableCommand.Type

    func parse(arguments: [String]) throws -> FBParsableCommand {
        var argumentDict = parseArguments(arguments)
        argumentDict.merge(defaultParameters()) { left, right in left } // merging
        let decoder = FBOptionsDecoder(arguments: argumentDict)
        return try command.init(from: decoder)
    }
}

Thus when an optional option is used:

struct FBRepeat: FBParsableCommand {

    @FBOption
    var requiredPhrase: String

    @FBOption
    var optionalPhrase: String = "hello"

    mutating func run() throws {
        for i in 1...2 {
            print(requiredPhrase)
            print(optionalPhrase)
        }
    }
}

FBRepeat.main()

The @FBOption wrapper will be decoded with its default value:

> fbrepeat --requiredPhrase bonjour
bonjour
hello
bonjour
hello

The decoding is fully functional!

Let’s implement the final feature: generating a help description when requested.

Generating command help

Thanks to the mirror inspection already in place, it will be easy. We just have to add some final touches to our API.

We add a FBCommandConfiguration structure so the clients can provide a description of their commands:

public struct FBCommandConfiguration {

    public let usage: String

    public init(usage: String) {
        self.usage = usage
    }
}

public protocol FBParsableCommand: Decodable {

    static var configuration: FBParsableCommandConfiguration { get }

    init()
    func run() throws
}

public extension FBParsableCommand {

    static var configuration: FBParsableCommandConfiguration {
        FBParsableCommandConfiguration(
            name: String(describing: Self.self).lowercased(),
            usage: ""
        )
    }
}

And a help parameter in each argument:

struct FBOptionDefinition {
    let isOptional: Bool
    let defaultValue: String?
    let help: String?
}

extension FBOption {

    public init(initialValue: String, help: String? = nil) {
        self.state = .definition(FBOptionDefinition(isOptional: true, defaultValue: initialValue, help: help))
    }

    public init(help: String? = nil) {
        self.state = .definition(FBOptionDefinition(isOptional: false, defaultValue: nil, help: help))
    }
}

Based on the static definition of a command, we can now easily generate a detailed error anytime we catch a help argument:

struct HelpError: LocalizedError {
    let localizedDescription: String
}

struct HelpGenerator {

    let description: String

    init(_ command: FBParsableCommand.Type) {
        var description = ""
        description += command.configuration.name
        description += "\n"
        if !command.configuration.usage.isEmpty {
            description += command.configuration.usage
            description += "\n"
        }
        for (key, definition) in command.argumentDefinitions() {
            description += "--" + key
            if let help = definition.help {
                description += ": " + help
            }
            if definition.isOptional {
                description += " (optional)"
            }
            if let value = definition.defaultValue {
                description += " (default \(value))"
            }
            description += "\n"
        }
        self.description = description
    }
}

struct FBCommandParser {

    let command: FBParsableCommand.Type

    func parse(arguments: [String]) throws -> FBParsableCommand {
        try checkForHelp(in: arguments)
        // ...
        return try command.init(from: decoder)
    }

    // ...

    private func checkForHelp(in arguments: [String]) throws {
        let helpIndicators = ["-h", "--help", "help"]
        let requestsHelp = helpIndicators.contains { indicator in arguments.contains(indicator) }
        if requestsHelp {
            throw HelpError(localizedDescription: HelpGenerator(command))
        }
    }
}

And here we go! FBSwiftArgumentParser is able to generate a description of our command when a help option is specified:

> fbrepeat --help
fbrepeat
--optionalPhrase (optional) (default Hello)
--requiredPhrase

The drawback of the SwiftParserArgument implementation

Our basic version of SwiftArgumentParser is small but it highlighted the main key tricks to create such a straightforward Swift library. Apple used a clever interlacing of operations executed at both compilation time and runtime.

At compilation time, all the command decoding code is generated on the client side thanks to the automatic conformance to Decodable. At runtime, SwiftArgumentParser parses the static definition of each command and adapts the decoding strategy consequently. This last step is permitted by an extended usage of property wrappers: it allows Apple to encapsulate and inject logic right inside the client objects declaration. I really liked it.

However, to use the API of SwiftArgumentParser, clients have to follow an unusual list of agreements to ensure the program runs smoothly:

Such agreements can not be guaranteed by the Swift compiler. In fact, SwiftArgumentParser counts more than twenty fatalError in its codebase. That is more than twenty manual checks that will lead to a crash if you are not respecting the usage contract.

That is why its implementation is hard to understand. In a regular codebase, targetting a production environment, we prefer to rely on the Swift compiler and therefore stay in the arguably safer environment it creates for us even if it means a more verbose code.

A Swift mod

With SwiftArgumentParser, Apple obviously made a tradeoff and decided that straightforwardness takes precedence over safety when writing CLI tools.

The fact that SwiftArgumentParser will always be at the top of any programming stack surely influenced the decision.

It reminds me of SwiftUI. It has a beautiful Swift API but it hides a lot of unexpected behaviors.

Like SwiftUI, SwiftArgumentParser adds its own programming paradigm on top of Swift. By establishing rules that go way beyond traditional Swift library usage restrictions, like “commands should not use regular properties”, it extends the language itself, creating sort of a mod of Swift and raising our code to new heights.

Of course, as the Swift compiler only knows Swift principles, those libraries have to validate their own rules at runtime with technics using metaprogramming as reflection. It inevitably leads to unsafe misuses.

In 2021, Apple had to demystify SwiftUI. By revealing a bit of the SwiftUI implementation, Apple somehow agreed its usage of Swift can be confusing. We actually could not write such libraries using regular Swift - at least, using Swift as Apple wants us to use it, in a safe manner.

Still, chances are, those libraries are pioneers in the language features development rather than classic unsafe implementations. I will be not be suprised to see new Swift mods from Apple in the future.

As a side note, it is interesting to see how, after putting so much effort in building a safe language, Apple allows itself to bypass some of the compiler checks and write “flexible” Swift code, close to the good old dynamic typing of Objective-C.

What’s next?

You can find all the code in a dedicated repository alongside a more detailed version of it.

I discovered SwiftArgumentParser when developing xcresource. You can take a look at it! It aims to facilitate downloading Xcode templates or snippets from git repositories.

Thanks for reading!