#210 (Forums Discussion)

Note This is an alternative implementation to #335, with the only notable difference being that extensions to nested external types are organized in a hierarchy.

Summary

This PR enables users of DocC to document extensions to external types, i.e. local extensions, that extend a type declared in a different module. These extensions are presented as part of the extending module's documentation archive.

All extensions to one type are merged into an Extended Type page, where "Type" can be either of struct, class, enum, protocol, or just "Type", if the extended type's kind is unknown. Extended Type pages list their children, conformances, etc. just as pages for normal types.

All top-level Extended Type pages, where the extended type was declared in the same module, are children of the same Extended Module page. The Extended Module pages are top level pages, i.e. by default listed in the archive overview page.

Referencing

Extended Module symbols are referenced using ``ModuleName``, Extended Type symbols via ``ModuleName/Path/To/ExtendedType``. The Path/To/ExtendedType has more than one segment for extensions to nested external types. Extensions to type aliases use the name of the aliased (original) type.

Please refer to SwiftDocC.docc/SwiftDocC/LinkResolution.md for a more detailed explanation.

The new pages can be curated and extended as usual using these references.

Documentation Aggregation

Extended Module pages have no documentation by default. It is possible to add documentation via extension markup documents.

Extended Type pages have default documentation. They use the longest documentation comment of any extension block that contributed to the extended type. Again, it is possible to add documentation via extension markup documents.

Example:

/// This will be discarded.
public extension String { /* ... */ }

/// This will be the documentation for ``Swift/String``, as
/// it is the longest (in terms of numbers of lines) documentation comment
/// above an extension to `String` that is defined in this module.
public extension String { /* ... */ }

Screenshots

The list of Extended Modules in the archive overview.

An example for an Extended Module page.

An example for an Extended Structure page.

The bottom part of the same Extended Structure page.

Implementation

Many of the files only had to be changed to introduce the six new symbol kinds module.extension, class.extension, struct.extension, enum.extension, protocol.extension, and unknown.extension as well as the new declaredIn and inContextOf relationship kinds.

Other than that, there are three components where actual logic had to be adapted:

SymbolGraphLoader

The loader detects whether or not the loaded symbol graphs contain swift.extension symbols. If all (non-main) files contain swift.extension symbols, the ExtendedTypesFormatTransformation is applied to (non-main) symbol graphs, and the loader configures the GraphCollector to merge extension graphs with the extending main graph instead of the extended one. If the loader detects a mixed input where some SGFs use the extension block format and some don't, it aborts loading with an error.

If there is no SFG that uses the extension block format, the loader behaves as it did prior to this PR.

ExtendedTypesFormatTransformation

This transformation generates the swift.module.extension and swift.TYPE_KIND.extension symbols from the swift.extension symbols. There is a detailed explanation available in code here.

The transformation essentially restructures and aggregates the swift.extension symbols to match the page structure we want to have in the end. After the transformation, most of Swift-DocC's logic can handle the new content without changes.

DocumentationContentRenderer+Swift -> navigatorTitle

Extension declarations only appear in top level code. Thus, if a nested external type is extended, it appears outside the context of its parent and therefore uses dots in its name, e.g. Outer.Inner. This behavior is reflected in the declaration fragments. The renderer could not handle dots in type names before, which resulted in the coloring being off. A more sophisticated parsing logic was added which is capable of handling names with dot infixes.

Note RenderNodeTranslator+Swift was removed because it was a copy of DocumentationContentRenderer+Swift where only the latter was used in production code and only the former was used in tests.

Dependencies

• apple/swift#59047
• apple/swift-docc-symbolkit#45
• apple/swift-docc-symbolkit#39
• apple/swift-docc-symbolkit#48
• apple/swift-docc-symbolkit#52

Testing

There exist unit tests that: - test detection of the Extension Block Symbol Graph Format and application of the ExtendedTypesFormatTransformation in SymbolGraphLoader - test the ExtendedTypesFormatTransformation - test references work with the old DocumentCacheBasedLinkResolver as well as the new PathHierarchyBasedLinkResolver.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [x] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary

Bug/issue #, if applicable: rdar://87784021

Summary

The test LogHandleTests.testFlushesStandardOutput() fails when run outside of Xcode because stdout is written to by other tests and types. This was breaking the assumption that stdout would be empty when it was written to by the test. This PR removes several print statements from the tests that have no effect on the tests, and changes the CoverageAction type to write to a provided log handle rather than print directly to stdout.

Dependencies

None.

Testing

Steps:

1. Download the latest 5.6 toolchain snapshot from swift.org.
2. Install the toolchain and cd </path/to/swift-docc>
3. Run xcrun --toolchain org.swift.<toolchain ID> swift test (i.e. xcrun --toolchain org.swift.56202203021a swift test)
   • You can get the bundle ID of a toolchain on macOS with the command /usr/libexec/PlistBuddy -c "Print CFBundleIdentifier:" /Library/Developer/Toolchains/<toolchain>/Info.plist
4. Verify that no test failures are reported.
5. Verify that the output is formatted as expected when run with the --experimental-documentation-coverage flag (i.e. swift run docc convert --experimental-documentation-coverage --level=brief /path/to/docs.docc)

Checklist

• [x] ~Added~ Updated tests
• [x] Ran the ./bin/test script and it succeeded
• ~Updated documentation if necessary~

Bug/issue #, if applicable: SR-15353

Summary

This PR is meant to help debug SR-15353 in CI, to figure out why the testSerialConcurrentPerform test is periodically failing on one of the Linux builders. Right now, it just tweaks the assertion logging, but i have a hunch as to what's going on (resolution/precision issues in the systemUptime clock) that i'll implement when i have a better picture of what's going wrong on the CI machine. (I couldn't reproduce it locally, whether in macOS or in a Linux Docker container.)

Dependencies

N/A

Testing

This PR updates an existing test and doesn't touch any library code; behavior should not change.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [ n/a ] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [ n/a ] Updated documentation if necessary

While reading abut docc one learns about directives and @Image so one wants to use it... yet it does not seem to work in Articles, and one has to resort to the markdown method ![]() this is rather confusing, as no warning is issued about the "wrong" use or anything else, leading one to dig through and start guessing if one messed up paths, names or something else.

Expected behavior

Option 1: Can we make @Image(source: work in articles, in addition to the markdown style?

Option 2: Actually banning @Image could be an option as in Articles it might be better to use ![]()? But then we should issue warnings when this not supported use is detected.

Actual behavior

When using @Image(source: "example.png", alt: "...") nothing happens, not even an empty <img> is rendered.

Steps to Reproduce

Make some article, and use @Image in it; I did so in the following PR and it did not work: https://github.com/apple/swift-distributed-actors/pull/929/commits/0262b759738b4c92aa6ca10fd89d398f53086159 (open source package)

rdar://94284119

| | | |------------------|-----------------| |Previous ID | SR-15410 | |Radar | rdar://63987302 | |Original Reporter | rnikander (JIRA User) | |Type | Bug |

Attachment: Download

Issue Description:

I have a Swift package in which I define some extensions on matrix types like simd_float3x3. I tried to use DocC to create documentation for this package, but I can't get it to show the extensions.

Example of code in my module that I can't document:

extension simd_float3x3 {
    /// Format the matrix with a default style
    public func formatted() -> String { ... }

I guessed at some syntax in the markdown files. For example: ``simd_float3x3/formatted``. But I'm not familiar enough with DocC yet to have any suggestions on what the syntax should be.

| | | |------------------|-----------------| |Previous ID | SR-15572 | |Radar | rdar://problem/86279414 | |Original Reporter | @ethan-kusters | |Type | Improvement | |Status | Resolved | |Resolution | Done |

Issue Description:

Today Swift-DocC always renders a framework's more technical name as the title of the framework overview page. In many scenarios, developer's will likely want to customize this value when they're not limited by what characters are valid in the framework's actual name.

For example, the SwiftDocC framework, may prefer to render as "Swift DocC" or "Swift-DocC". Or maybe "ArgumentParser" would prefer to render as "Swift Argument Parser".

Swift-DocC should expose customization on the CLI and in documentation catalogs for a module's display name.

Bug/issue #, if applicable: SR-15839 Forum discussion: https://forums.swift.org/t/can-we-link-and-render-external-images-that-is-from-urls

Summary

Add a new ExternalMediaResolver to support external media reference.

Dependencies

None

Testing

Test external image and video support in docs and articles.

Checklist

• [x] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary

TODO

• [x] Currently this PR does not support external media in tutorials. Will solve it in a follow-up PR.

Add snippet support

• Add a @Snippet block directive that takes one argument: the path to the snippet as if it were a symbol link.
• Render a snippet as follows:
  • The snippet explanation
  • For each snippet chunk:
    • The name in bold if present
    • The source code as a code block

Snippets are transmitted to DocC via the Symbol Graph format.

Snippets do not have their own dedicated page as they won't have accompanying long-form content. Snippets may be collected in a snippet "index" where one can easily search or browse for them in one place, where they'll be grouped appropriately.

This requires https://github.com/apple/swift-docc-symbolkit/pull/10 (now merged)

This also requires https://github.com/apple/swift-docc-symbolkit/pull/15.

Feature/SR-15311

Summary

Swift-DocC currently ignores SVG and GIF files.

This PR add support for recognizing them as images in the same we do for JPEG, JPG, and PNG files.

Testing

Steps:

1. Download sample project developer.apple.com/documentation/xcode/slothcreator_building_docc_documentation_in_xcode
2. Download a svg file and a gif file and drag them to the Package's Sources/SlothCreator/SlothCreator.docc/Resources folder
3. Replace the content in SlothCreator.tutorial

@Image(source: slothcreator-intro.png, alt: "An illustration of ... tutorials.")

to

@Image(source: xx.svg, alt: "An illustration of ... tutorials.")

@Image(source: xx.gif, alt: "An illustration of ... tutorials.")

4. Build documentation to verify.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [x] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary

Bug/issue #, if applicable: rdar://59660520

Summary

This adds a mechanism for sorting and filtering strings based on how "similar" they are to a specific string for use to create near-miss suggestions in diagnostics.

The changes are divided in two parts:

• One that wraps the the difference capabilities in Swift to calculate the length of the segments of common and difference elements.
• Another that "score" these segments to "rank" values based on how "similar" they are to the authored value.

Most of changes in this PR (1500+ LOC) are the tests for both these parts.

For the second part there is no single right or wrong answer for how to do that. The approach I took is to write test of what results I would expect when matching a collection of possibilities against a certain value and then tweak the "score" and "ranking" implementations until they produces the results that I'd expect. This implementation is completely arbitrary.

My hopes is that when there are situations where this doesn't behave well, we'll add those as more tests and tweak the "score" and "ranking" implementations again.

Dependencies

None

Testing

Please give this a spin with some real code bases to see how it performs with real links in real projects.

To do so:

1. Checkout this branch and build the docc executable.
2. Build documentation for another project with DocCUseHierarchyBasedLinkResolver enabled and DOCC_EXEC pointed to the locally built docc executable.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [x] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary

Bug/issue #, if applicable: Close #363

Summary

Add a new flag "--warnings-as-errors" for "docc convert" Command.

If this flag is set, the engine will treat the warning severity level as an error severity level

Forum discussion post: https://forums.swift.org/t/add-support-for-warnings-as-errors/59832

Dependencies

None

Testing

1. git clone https://github.com/apple/swift-markdown.git
2. Generate swift-markdown's symbol-graph
3. Run

docc convert swift-markdown/Sources/Markdown/Markdown.docc`
--fallback-display-name SwiftDocC
--fallback-bundle-identifier org.swift.SwiftDocC 
--fallback-bundle-version 1.0.0
--additional-symbol-graph-dir xx/swift-markdown.build/Debug/Markdown.build/symbol-graph

and see a list of warnings. 4. Run

docc convert swift-markdown/Sources/Markdown/Markdown.docc`
--fallback-display-name SwiftDocC
--fallback-bundle-identifier org.swift.SwiftDocC 
--fallback-bundle-version 1.0.0
--additional-symbol-graph-dir xx/swift-markdown.build/Debug/Markdown.build/symbol-graph
--warnings-as-errors

and see all the warnings are treated as errors.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [x] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary

Bug/issue #, if applicable: #438

Summary

This PR adds fixits to the warnings generated by reference resolution for near-miss and collision scenarios.

The fixits are always generated so that they replace the whole reference, i.e. the replacement area includes the whole reference link, plus the surrounding backticks.

Fixits only fix the first faulty segment in a reference. They do not alter the valid prefix and keep the unchecked suffix in place.

For example:

• /MyKit/MyClas -> /MyKit/MyClass
• MyClas -> MyClass
• MyClas/validMember -> MyClass/validMember
• MyClas/invalidMember -> MyClass/invalidMember -> MyClass/validMember (the second fixit only becomes available after running docc again)

The implementation is structured as follows:

• the actual replacement strings are generated in PathHierarchy/Error/replacements(for:)
• PathHierarchyBasedLinkResolver/resolve(_:in:fromSymbolLink:context:) calls PathHierarchy/Error/replacements(for:) and attaches them as candidates to the UnresolvedTopicReference returned in the TopicReferenceResolutionResult/failure(_:errorMessage:) (Adding the candidates field to UnresolvedTopicReference avoids a breaking change in TopicReferenceResolutionResult/failure(_:errorMessage:)).
• MarkupReferenceResolver and ReferenceResolver pass the candidates to unresolvedReferenceProblem(reference:source:range:severity:uncuratedArticleMatch:underlyingErrorMessage:candidates:), which then generates Solutions if there are candidates available and attaches them to the Problem

Beyond that I had to make minor adaptions to make sure range offsets are applied not only to a Problem's diagnostic, but also to possibleSolutions.

Dependencies

None.

Testing

Updated SymbolTests/testUnresolvedReferenceWarningsInDocumentationExtension() to also test fixits and added a couple of test-cases relevant for fixits.

Make sure to set DOCC_USE_HIERARCHY_BASED_LINK_RESOLVER=YES when running the tests.

For manual testing also make sure to enable fixits with --emit-fixits, e.g.: DOCC_USE_HIERARCHY_BASED_LINK_RESOLVER=YES .build/debug/docc convert Sources/SwiftDocC/SwiftDocC.docc --emit-fixits --additional-symbol-graph-dir .build/arm64-apple-macosx/symbolgraph

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [x] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary

Description

When generating our SDK docs with DocC, classes marked as unavailable in Application Extensions are marked as deprecated in the generated docs.

Steps To Reproduce

Generated DocC page for PaymentSheet.FlowController Class definition for PaymentSheet.FlowController

If we define our class:

@available(iOSApplicationExtension, unavailable)
@available(macCatalystApplicationExtension, unavailable)
public class MyClass: NSObject { }

The Swift compiler will output this in the symbol graph JSON:

"availability":[{"domain":"macCatalystAppExtension","isUnconditionallyUnavailable":true},{"domain":"iOSAppExtension","isUnconditionallyUnavailable":true}]

And Swift-DocC will mark it as deprecated in the docs data JSON:

{"deprecated":true,"role":"symbol","title":"MyClass","fragments":[...]}

I think the issue is that the AvailabilityParser marks a symbol as deprecated if every Availability entry for a symbol is marked as isUnconditionallyUnavailable, even if the symbol is implicitly available in other domains. This final check seems incorrect:

        // Check if the symbol is unconditionally deprecated
        return availability.availability
            .allSatisfy { $0.isUnconditionallyDeprecated || $0.isUnconditionallyUnavailable || $0.deprecatedVersion != nil }

DocC already checks if a "universal deprecation" exists (a deprecation where domain == nil) just above that line, though I think it should also check if the domain is *.

Checklist

• [ ] If possible, I've reproduced the issue using the main branch of this package.
• [X] This issue hasn't been addressed in an existing GitHub issue.

Expected Behavior

DocC marks the iOSApplicationExtension-unavailable class as Deprecated with a strikethrough effect.

Actual behavior

DocC marks it as available, or maybe marks it as "not available in Application Extensions".

Swift-DocC Version Information

Bundled in Xcode 16.1

Swift Compiler Version Information

swift-driver version: 1.62.15 Apple Swift version 5.7.1 (swiftlang-5.7.1.135.3 clang-1400.0.29.51)
Target: arm64-apple-macosx13.0

Bug/issue #, if applicable:

Summary

This fixes two minor warnings in the code

• A deprecation warning in a test (after sha512Checksum was renamed to checksum)
• A warning about the inferred type of an empty array literal

Dependencies

n/a

Testing

n/a

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [ ] Added tests
• [x] Ran the ./bin/test script and it succeeded
• [ ] Updated documentation if necessary

Bug/issue #, if applicable:

Summary

This updates the current option directives to follow the naming convention that's proposed in this forum post.

Dependencies

n/a

Testing

Use the AutomaticArticleSubheading, AutomaticSeeAlso, and AutomaticTitleHeading option directives in content.

• The syntax for disabling the default behaviors should remain the same as before.
• The syntax for redundantly enabling these features now use an explicit "enabled" value.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [x] ~Added~ Updated tests
• [x] Ran the ./bin/test script and it succeeded
• [x] Updated documentation if necessary

Bug/issue #, if applicable:

Summary

I found typos in some places, so fixed them. please correct me if I am wrong.

Dependencies

None

Testing

Please check the spelling

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

• [ ] ~Added tests~
• [x] Ran the ./bin/test script and it succeeded
• [ ] ~Updated documentation if necessary~

Feature Name

Combine multiple code language for export

Description

We need to export Swift and Objc documentation for a single file to host in our webpage.

Motivation

We have a lot of libraries with both swift and objective-c code, we need to be able to export both language documentation in a single file to host in our webpage.

Importance

No response

Alternatives Considered

No response