The Symbol.swift currently has 1400 lines.

This PR is intended to split the Symbol.swift file into a file system hierarchy.

So that the Symbol.swift only have SymbolGraph.Symbol definition and also makes it easier for writing test case.

Discussion

• [x] Which style do you prefer?

- Symbol
	- Availability
		- Availability.swift
		- Domain.swift
	- Symbol.swift  	

Or

- Symbol
	- Availability
		- Domain.swift
	- Availability.swift
- Symbol.swift  	

• [x] What should we do if there are 2 struct/enum both named as "Kind"

Currently there are SymbolGraph.Symbol.DeclarationFragments.Fragment.Kind and SymbolGraph.Symbol.Kind.

Although they are not in the same folder, they are in the same module. Which is not allowed by Swift

error: filename "Kind.swift" used twice: 'Symbol/DeclarationFragments/Fragment/Kind.swift' and 'SymbolKit/SymbolGraph/Symbol/Kind.swift'
note: filenames are used to distinguish private declarations with the same name

https://belkadan.com/blog/2021/11/Swift-Mangling-Regret-Private-Discriminators/

Status

• [x] ~WIP~ After the above 2 discussion is solved, I'll complete the full PR.

Summary

This sets the foundation for deploying Swift SymbolKit's documentation to GitHub pages.

• Adds the Swift-DocC Plugin as a dependency of Swift-SymbolKit.
• Adds a script to simplify deployment of documentation to GitHub pages.
• Adds check-source and test script and fix missing headers.
• Update README docs (Remove some duplication and add a link to the github page).
• Adds pull request template file as other swift-docc related repo.

After merging this PR, we should be able to prepare a gh-pages branch on the main swift-docc-symbolkit repo with no content on it, run the bin/update-gh-pages-documentation-site script and have docs available.

Dependencies

None.

Testing

With Swift 5.6, run:

swift package --disable-sandbox preview-documentation --target SymbolKit

and confirm the Swift-DocC preview works as expected.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [x] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary

Most of the content of this PR comes from @ethan-kusters. This is just a port for swift-docc-symbolkit repo. See more info here https://github.com/apple/swift-markdown/pull/32

This was discussed lightly in #8 after it was merged.

Shared prefixes lead to long file names that are difficult to tell apart. We should only use prefixes that derive from parent type/module etc names without trailing "+"s when there is a naming clash. (Ideally, SwiftPM should be able to handle same-name files in different paths, but that's outside the scope of this PR.) "+" should be reserved for protocol conformances, such as "SemanticVersion+Comparable", which I believe is the convention in all other official open-source projects.

Currently each nested type definition is defined in its own file. I don't think this is necessary, and types that are closely related can be grouped in the same file. But that can be a follow-up PR.

This PR is required to allow SwiftDocC to restructure the Symbol Graph Format after decoding the Symbol Graph Files. This is necessary to implement extensions to external types (fix apple/swift-docc#210).

Summary

This PR contains four major changes:

• change Symbol.KindIdentifier to use a struct-type with static properties for known values
• allow for registering new Symbol.KindIdentifiers to ensure identifiers get parsed correctly
• change Codable conformance of Symbol and Relationship to allow for encoding and decoding unknown Mixins
• change Hashable conformance of Relationship so unknown Mixins can be taken into account while hashing

Please refer to SymbolKit.docc/GraphFormatExtensions.md to see how the new APIs are to be used.

The most critical part of this implementation is how to handle decoding (and encoding) of custom Symbol.KindIdentifiers and Mixins. Symbol (and Relationship's) Codable implementations must know how to encode/decode the custom types. This PR solves this problem by having the instance defining the custom extensions register the extensions to the Encoder/Decoder. This is realized via Encoder/Decoder's userInfo. This choice has two key advantages over registering the extensions to the static context:

• no need for synchronization
• different extension views on the symbol graph format can coexist within one executable without influencing each other

Next to the user info based approach, this PR also offers the option to register Symbol.KindIdentifier to the static context for convenience.

#7 only provided a solution for Symbol.KindIdentifier which did not register custom identifiers in a global context, but stored the raw value in the Symbol.KindIdentifier locally. This, however, has one disadvantage: Custom identifiers are parsed differently than those defined by SymbolKit. The language prefix cannot be removed, since the initializer cannot know, if the first section of the identifier is a language, or already important for the language-agnostic identifier.

Dependencies

This PR should be merged after #45 as this PR is based on that branch.

Furthermore, this PR incorporates similar changes as #7 which seems to be stale.

Testing

I added test cases that:

• check encoding/decoding works with custom mixins
• Symbol.KindIdentifiers registered on the static context are parsed as expected and appear in allCases
• Symbol.KindIdentifiers registered on a decoder are decoded as expected and do not appear in allCases
• Hashable conformance of Relationships works as intended with unknown Mixins which do or do not conform to Hashable

Benchmark

Benchmarked this PR (e6478fd383a01e4f79421d9bbac5610bbffdc9dc) against main (cfa80d701f7d8699d64237c7bcc760d6c9616529) using Swift-DocC's bin/benchmark.swift. Swift-DocC was on main (ed1770f974c3a1c6d2d1437cc07bc90a8f220c89). I used a bundle generated using swift run make-test-bundle --sizeFactor 50.

There is no significant change in runtime:

+-----------------------------------------------------------------------------------------------------+
| Metric                                   | Change     | Before               | After                |
+-----------------------------------------------------------------------------------------------------+
| Duration for 'bundle-registration' (sec) | +0.54%     | 12.01                | 12.08                |
| Duration for 'convert-total-time' (sec)  | +0.28%     | 19.92                | 19.98                |
| Duration for 'documentation-processing'  | -0.15%     | 7.38                 | 7.37                 |
| Duration for 'finalize-navigation-index' | no change  | 0.24                 | 0.24                 |
| Peak memory footprint (MB)               | -0.21%     | 1966.29              | 1962.25              |
| Data subdirectory size (MB)              | no change  | 538.01               | 538.01               |
| Index subdirectory size (MB)             | no change  | 4.84                 | 4.84                 |
| Total DocC archive size (MB)             | no change  | 566.93               | 566.93               |
| Topic Anchor Checksum                    | no change  | 08d0a84bec913460905f | 08d0a84bec913460905f |
| Topic Graph Checksum                     | no change  | 74d25c4f1ee185ad5676 | 74d25c4f1ee185ad5676 |
+-----------------------------------------------------------------------------------------------------+

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [x] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary
• [x] Ran Benchmarks

Bug/issue #, if applicable: rdar://91246013

Summary

This PR updates the UnifiedSymbolGraph to also differentiate relationships by a Selector. Since relationships don't have an interfaceLanguage of their own, their respective symbols are searched to supply one. They are first searched in the graph being processed, then in the in-progress unified graph as a whole. The latter step is intended to catch protocol conformance on an external type when it's being loaded from an extension symbol graph.

Dependencies

While the change was made in a way that preserved source compatibility, a corresponding change to Swift-DocC (https://github.com/apple/swift-docc/pull/304) is required to use the new information.

Testing

A new test case is added to verify the relationship sorting behavior in a basic case. As end-to-end behavior is not fully implemented yet, a debugger and/or test project is required to observe the new behavior.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [x] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary

Snippets need a place to put the chunks of its code listing.

Chunks contain source code, and an optional source language and name for a subheading at the minimum.

Bug/issue #, if applicable: rdar://91237393

Summary

Adds support for building the SymbolKit library with library evolution enabled. This is disabled by default, and to enable the behavior, add an .enable-library-evolution file at the root of your checkout.

Dependencies

None.

Testing

Verify that SymbolKit is built with library evolution enabled by inspecting build logs if and only if the checkout has an .enable-library-evolution file at its root.

Checklist

• ~[ ] Added tests~ N/A
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary

This PR updates the images used in the README and the docs to allow for the doc catalog added in https://github.com/apple/swift-docc-symbolkit/pull/6 to have proper dark-mode images.

Bug/issue #, if applicable: rdar://91166981

Summary

To allow SymbolKit to load symbol graphs from the in-progress Clang symbol-graph support (cf. https://reviews.llvm.org/D122611 and https://reviews.llvm.org/D122446), two new symbol kinds need to be added: One for C preprocessor macros, and one for Objective-C interface variables. This PR also adds a new test that ensures that all the KindIdentifier cases can parse back as themselves, to ensure that the .init(identifier:) and lookupIdentifier() functions don't get out of sync.

Dependencies

None

Testing

The "ExtractAPI" symbol graph support in Clang is still experimental, but symbol graphs can be found as part of the ExtractAPI tests in the LLVM main branch. Loading either the macros.c output or the objc_interface.m output should result in a parsed SymbolGraph that has no .unknown symbols.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [x] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [ n/a ] Updated documentation if necessary

SE-0356 latest changes include parsing "slices" out of snippets. These are not quite the same as the current "chunks", which were a speculation that eventually formed into "slices". Deprecate chunks and add the slice data model.

Move snippet presentation code to a line-based format: slices are represented simply as a name and index range to pull their code lines.

Add the isVirtual property to modules, for modules that are created implicitly to hold relationships. When snippet symbol graphs are created, a fake module is created to hold the snippets, since snippets do not come from any one module. This property simplifies what is basically just a heuristic in DocC to a simple factual check.

rdar://95220716

Required for related changes:

• https://github.com/apple/swift-docc/pull/338
• https://github.com/apple/swift-docc-plugin/pull/20

This adaption is required to fix apple/swift-docc#210

Summary

Adds a strategy option to GraphCollector's init. The ExtensionGraphAssociation option decides if extension graphs are merged with the extending or extended graph.

Dependencies

This PR does not introduce a breaking change and can be merged independently.

Testing

I added a new unit test to test the merging behavior for both ExtensionGraphAssociation options.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [x] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary

Bug/issue #, if applicable: None

Summary

When https://github.com/apple/swift-docc-symbolkit/pull/28 landed, the code for dump-unified-graph was not updated to use the new relationship data. This PR updates the Encodable extension on UnifiedSymbolGraph to use the relationshipsByLanguage and orphanRelationships properties instead of the old unified relationships property. This removes a deprecation warning when building SymbolKit.

Dependencies

None

Testing

Rerun the testing instructions for https://github.com/apple/swift-docc-symbolkit/pull/29. The Xcode 14 beta will generate multiple symbol graphs per-language, so passing the symbol-graph directory from a project's build folder to --symbol-graph-dir will work.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [ n/a ] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [ n/a ] Updated documentation if necessary

This is a correct but somewhat over-engineered reinplementation of SymbolGraph.SemanticVersion.

The most important (and also most invasive) changes are the handling of pre-release and build metadata. These include their validation and (their contribution to version-) comparison.

Some less important changes include the following:

• Version core identifiers are changed to be backed by UInt, because they’re not allowed be less than 0. This might appear to be a departure from the convention of using Int for even always-positive integers (e.g. Array.Index). However, in this specific case, the use of version coure identifiers is mostly contained in the domain of constructing semantic versions, and does not have any apparent need to participate in numeric operations outside of the domain other than comparing individual identifiers. And considering Swift’s current lack of build-time evaluation and error handling, using UInt looks like the right trade off between ergnomics and “safety”.

• Deprecated the initializer for a new one that throws error, so that erros can be handled instead of runtime trapping.

• Added some facilities for inspecting and working with versions’ semantics in general.

• ~Updated the tools version specifier in the main package manifest to “5.6”, because Swift 5.2 has some type-checking bugs that affect the new implementation.~

• Added a lot of tests, which should cover a large area of (edge) cases.

Checklist

• [x] Added tests
• [ ] Ran the ./bin/test script and it succeeded (Tests pass using the 2020-05-23 trunk toolchain on Xcode, but ./bin/test fails with "_$s14SymbolKitTests05ErrorC0C26testOversizedNumericValuesyyF3sumL_ySsSS_SStF10zeroPaddedL__8toLengthSaySJGSS_SitF", referenced from: _$s14SymbolKitTests05ErrorC0C26testOversizedNumericValuesyyF3sumL_ySsSS_SStF in ErrorTests.swift.o"
• [x] Updated documentation if necessary

| | | |------------------|-----------------| |Previous ID | SR-16111 | |Radar | None | |Original Reporter | @QuietMisdreavus | |Type | Improvement |

Issue Description:

SymbolKit currently defines optional data fields for symbols and relationships as a set of "mixins", which is represented as a map from a "mixin key" to the parsed value of that mixin. Swift-DocC uses these mixins for various uses, but checking for a value and reading it out is a bit cumbersome. For example, this is how Swift-DocC loads availability information from a symbol:

if var availability = symbol.mixins[SymbolGraph.Symbol.Availability.mixinKey] as? SymbolGraph.Symbol.Availability

This could be greatly simplified into some kind of getMixin method on symbols and relationships in SymbolKit. It could look something like this:

func getMixin<T>() -> T? where T: Mixin {
    self.mixins[T.mixinKey] as? T
}

In fact, Swift-DocC already defines something similar, as an extension on mixin dictionaries themselves:

extension Dictionary where Key == String, Value == Mixin {
    func getValueIfPresent<T>(for mixinType: T.Type) -> T? where T: Mixin {
        return self[mixinType.mixinKey] as? T
    }
}

Adding one of these methods to SymbolKit and encouraging its use throughout Swift-DocC would make its codebase much more readable.

| | | |------------------|-----------------| |Previous ID | SR-15982 | |Radar | rdar://problem/90348996 | |Original Reporter | @franklinsch | |Type | Improvement |

Issue Description:

SymbolKit decodes unknown Symbol mixins just fine by adding them to Symbol.mixins, but {{fatalError}}s when encoding unknown mixins: https://github.com/apple/swift-docc-symbolkit/blob/main/Sources/SymbolKit/SymbolGraph/Symbol/Symbol.swift#L180.

For example, when decoding a symbol graph from the Swift compiler (which includes the location mixin) and encoding it back, SymbolKit crashes.

We should make SymbolKit encode unknown mixins.

Resolves rdar://84276085

This PR changes the type definition of KindIdentifier from an enum into a struct with static fields for common values. This has two effects:

1. Symbol kinds that don't match the currently-known set can still be properly handled and parsed, by saving the identifier string from the symbol graph.
2. Adding new symbol kinds to the set of known kinds is no longer a breaking change for downstream users; matching KindIdentifier now requires a case default branch to handle unknown values, instead of simply matching on the .unknown enum case.

| | | |------------------|-----------------| |Previous ID | SR-15551 | |Radar | rdar://85982095 | |Original Reporter | @ethan-kusters | |Type | Improvement |

Issue Description:

SymbolKit should provide public API to access the available languages of both a symbol and a unified symbol graph.

We ran into the need for this in the work for mixed language support implemented here: https://github.com/apple/swift-docc/pull/47/files.