I'm noticing on the output of xcbuild that it will output the copy-header phases before performing a compile, this doesn't respect the ordering that Xcode has for a build or the ordering that the phases have in the project file.

Implements the module map features needed for #88. They're part of the grab bag of build things in ProductTypeResolver; that probably should be split up in a later PR. For now, this fixes that task and should be a big help for any Swift projects using CocoaPods.

share:xcbuild chen-pc$ make mkdir -p build cmake -Bbuild -H. -G Ninja -DCMAKE_INSTALL_PREFIX= CMake Error: CMake was unable to find a build program corresponding to "Ninja". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool. CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly. Missing variable is: CMAKE_C_COMPILER_ENV_VAR CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly. Missing variable is: CMAKE_C_COMPILER CMake Error: Could not find cmake module file: /Users/chen-pc/xcbuild/build/CMakeFiles/3.2.2/CMakeCCompiler.cmake CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly. Missing variable is: CMAKE_CXX_COMPILER_ENV_VAR CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly. Missing variable is: CMAKE_CXX_COMPILER CMake Error: Could not find cmake module file: /Users/chen-pc/xcbuild/build/CMakeFiles/3.2.2/CMakeCXXCompiler.cmake CMake Error at CMakeLists.txt:10 (project): No CMAKE_C_COMPILER could be found.

Tell CMake where to find the compiler by setting the CMake cache entry CMAKE_C_COMPILER to the full path to the compiler, or to the compiler name if it is in the PATH.

CMake Error at CMakeLists.txt:10 (project): No CMAKE_CXX_COMPILER could be found.

Tell CMake where to find the compiler by setting the CMake cache entry CMAKE_CXX_COMPILER to the full path to the compiler, or to the compiler name if it is in the PATH.

CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage -- Configuring incomplete, errors occurred!

Instead of relying on a constructor to grab the argc/argv (which doesn't work on my Centos7 machine), use the argc/argv passed in from main().

This also defines the global default context to whatever is created in each executable's main() function instead of creating a new object.

Test plan:

• run 'plutil' on my Linux machine (after adding "-lpthread" added to the build invocation)
• run 'plutil' on my OSX machine

See https://github.com/facebook/xcbuild/issues/138

While trying to setup the Lyft project to use xcbuild, I ran into some issues with dependencies. We use CocoaPods to integrated 3rd party, and some first party development dependencies. The structure looks something like this:

Main Project
  -> Alamofire
  -> Internal Library
       -> zipzap
  ...

In this simplified example, the issue we're running into is that zipzap cannot be found. This leads to errors in the Internal Library, and stops us from getting anywhere else.

I've attempted to reproduce the same issue in a sample project:

test-xcbuild4.zip

In this project if you run (with or without specifying -executor ninja):

$ xcbuild -executor ninja -workspace test-xcbuild4 -scheme test-xcbuild4

You will see that it fails to build with the error:

{
  "kind": "finished",
  "name": "compile",
  "pid": 48974,
  "output": "blah blah/AppDelegate.swift:10:8: error: no such module 'zipzap'\nimport zipzap\n       ^\n",
  "exit-status": 1
}

(here's the entire build log)

But if you build from within Xcode, everything compiles correctly.

I would love some advice for where to start when I'm facing errors like this, as I'd love to be able to set our project up to benefit from the speed of Ninja! ⚡

Thanks!

When building with xcbuild, I end up with dsymutil invocations like this:

GenerateDSYMFile /Users/martin/Library/Developer/Xcode/DerivedData/myproj-random/Build/Products/Release-iphoneos/example.app/example
    cd /Users/martin/code/example
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/dsymutil /Users/martin/Library/Developer/Xcode/DerivedData/myproj-random/Build/Products/Release-iphoneos/example.app/example -o 

I.e. the output filename isn't provided at all.

For some reason, when building from xcbuild on OS X, this doesn't actually print any error message or indicate any failure, but when this same command line is called manually, it prints an error like this:

dsymutil: for the -o option: requires a value!

When I try to use xcbuild on linux, the missing parameter to -o ends up being an issue.

$ ./build/xcbuild 
warning: unhandled Validation key AdditionalValidationClass
error: registering Architecture specification 'Standard' in domain iphonesimulator twice
error: registering Architecture specification 'i386' in domain iphonesimulator twice
error: registering Architecture specification 'x86_64' in domain iphonesimulator twice
error: registering Architecture specification 'Standard32_bit' in domain iphonesimulator twice
error: registering Architecture specification 'Universal64bit' in domain iphonesimulator twice
error: registering Architecture specification 'Native' in domain iphonesimulator twice
error: registering Architecture specification 'Standard' in domain iphoneos twice
error: registering Architecture specification 'armv7' in domain iphoneos twice
error: registering Architecture specification 'armv7s' in domain iphoneos twice
error: registering Architecture specification 'arm64' in domain iphoneos twice
error: registering Architecture specification 'Platform32_bit' in domain iphoneos twice
error: registering Architecture specification 'Platform64_bit' in domain iphoneos twice
error: registering Architecture specification 'Platform32_64_bit' in domain iphoneos twice
error: registering BuildSystem specification 'com.apple.build-system.native' in domain iphoneos twice
[1]    47231 segmentation fault  ./build/xcbuild

I don't solve this problem

xcrun attempts to determine the developer root so that it knows where how to locate toolchains and sdks. Currently the developer root is resolved through the following steps

1. Checking DEVELOPER_DIR environment variable
2. Check the /var/db/xcode_select_link
3. /usr/share/xcode-select/xcode_dir_path (1) suffers from having to configure this env var everytime xcrun is run. (2) and (3) suffers on needing to be root to configure on most systems.

This change adds another symlink to check for: ~/.xcsdk/xcode_dir_path

Test Plan:

1. Create symlink ~/.xcsdk/xcode_dir_path
2. xcrun -v -l --sdk test-sdk -f test-script

Change-Id: I88ed516e7264cffa263f6ed831e34083930e3337

I noticed that some of the property list files being built by xcbuild are not formatted correctly. I found a few example property lists that show generic examples that most developers will try to use in preprocessing of these files. In general this preprocessor works as a C Preprocessor with a few quirks added. Some notes on this can be found on Apple Technical Note TN2175.

In general I would assume that the plist utility will support the following operations...

• Pre-processor Conditionals: #if, #ifdef, #ifndef, #else, #elif and #endif.
• C-style Relational Operators ( see: Wikipedia: Operators in C and C++ )
• C-style Logical Operators ( see: Wikipedia: Operators in C and C++ )
• C-Style comments /* Comment */
• C++-Style comments // Comment ( when -traditional is not set )

Example 1: IOAudioFamily version 204.3 This is the one of the simplest examples of plist editing. All it does is define a few variables in an xcode configuration file and output it in the kernel extensions PLIST file..

Example 2: IOHIDFamily version 701.20.10 A slightly more complex example. This configures multiple property lists.

Example 3: IOStorageFamily version 191 This includes an example of preprocessor events to remove sections of the property list on a Embedded platform where TARGET_OS_EMBEDDED is defined in the TargetConditionals header.

Example 4: Some more Pre-processor examples. For the most part if it's in the TargetConditionals header for OSX, it'll appear in the Property list utility.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>PreProcess_KEYS</key>
    <dict>
        <key>IsEmbedded</key>
#if !TARGET_OS_EMBEDDED
        <true/>
#else 
        <false/>
#endif

        <key>IsMac</key>
#if TARGET_OS_MAC
        <true/>
#else 
        <false/>
#endif
        <key>IsPowerPC</key>
#if TARGET_CPU_PPC
        <true/>
#else 
        <false/>
#endif
    </dict>
</dict>

Split out of #88.

@keith wrote:

Ok so as far as I can tell, based on your branch, the original issue here is fixed. If this next issue doesn't sound related to you, feel free to close this one and I can open a different one as needed!

The next issue I've ran into, which I can't yet tell if it's related, is that using xcodebuild it seems like (for better or worse) many imports are implicitly added. For example we have files like this:

$ cat Foo.swift
extension UIView {
   ...
}

Where we have no explicit import UIKit, yet these work somehow. I think some of this might have to do with other files that are importing 3rd party dependencies, and maybe those imports are polluting the entire modules namespace. Another similar case we have is:

$ cat Foo.swift
import UIKit

... NSBundle ...

Which will also fail to compile because of a missing import to Foundation. Although it seems like in other places, import UIKit will implicitly make the Foundation import. Again I'm not a fan of this behavior, but it has apparently been something we're relying on. I'm currently adding all the missing imports by running xcbuild until it fails, copying the command that fails, running it, fixing the imports, repeat. Also as a note, these don't get surfaced in xcpretty, which would probably make this easier, instead I've been grepping the log for exit-status": 1

Let me know if you think this is at all related.

@keith wrote:

Ok so after finding all these missing imports, it seems like that issue only applies to the non-app dependencies of our project. I'm not sure exactly what the difference is there, but I only had to modify imports in our first party dependencies, some from CocoaPods and some just custom frameworks in our main project, but none of our main project files. There could be a difference here since the main project is the only place with a bridging header, which might be a slightly more explicit way of polluting the global namespace 😬

@grp wrote:

Hm, very interesting! It would be useful to see a comparison of the swiftc invocations between xcbuild and xcodebuild for a target hitting the issue. Hard to imagine what could cause a difference like that, though!

• Separate out PNG loading into a separate module.
• Refactor PNG loading to be less tied to asset compliation.
• Add an implementation of PNG loading using ImageIO when available.

Hi Team

I am trying to use xcbuild in CentOS (x86_64 & ) to build react native project, request you to please provide below answers -

1.) Can we use xcbuild to build iOS react native projects as in my view Xcode's xcodebuild tool is part of the Xcode SDK from Apple — it's only available for download on Mac OS X.

2. Please help and provide how to archive and sign IPA file.

Looking to hear from you.

With Regards -Deepak

Hey,

I’m using bazel, which uses xcodebuild to determine the system version of various sdks. Although with xcbuild installed (with nix) I get this error:

xcode_configure.bzl:89:9: Invoking xcodebuild failed, developer dir: /Applications/Xcode.app/Contents/Developer ,return code 1, stderr: error: missing argument value for argument -sdk

It’s calling:

xcrun xcodebuild -version -sdk

Apple’s xcodebuild doesn’t require an additional value after sdk. Without that setting it won’t output the SDKs, and the additional arguments just mean it outputs specific SDKs only.

I believe this error is invalidating the build cache of anything depending on that information every time I rebuild.

[10/524] Building CXX object Libraries/acdriver/CMakeFiles/acdriver.dir/Sources/Compile/AppIconSet.cpp.o FAILED: Libraries/acdriver/CMakeFiles/acdriver.dir/Sources/Compile/AppIconSet.cpp.o /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -Dacdriver_EXPORTS -I../Libraries/acdriver/Headers -I../Libraries/xcassets/Headers -I../Libraries/libutil/Headers -I../Libraries/ext/Headers -I../Libraries/plist/Headers -I../Libraries/process/Headers -I../Libraries/graphics/Headers -I../Libraries/dependency/Headers -I../Libraries/libbom/Headers -I../Libraries/libcar/Headers -std=c++11 -fno-rtti -fno-exceptions -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -fPIC -Wall -Werror -fcolor-diagnostics -MD -MT Libraries/acdriver/CMakeFiles/acdriver.dir/Sources/Compile/AppIconSet.cpp.o -MF Libraries/acdriver/CMakeFiles/acdriver.dir/Sources/Compile/AppIconSet.cpp.o.d -o Libraries/acdriver/CMakeFiles/acdriver.dir/Sources/Compile/AppIconSet.cpp.o -c ../Libraries/acdriver/Sources/Compile/AppIconSet.cpp In file included from ../Libraries/acdriver/Sources/Compile/AppIconSet.cpp:9: In file included from ../Libraries/acdriver/Headers/acdriver/Compile/AppIconSet.h:12: In file included from ../Libraries/xcassets/Headers/xcassets/Asset/Asset.h:15: ../Libraries/libutil/Headers/libutil/Base.h:24:32: error: 'ptr_fun<int, int>' is deprecated [-Werror,-Wdeprecated-declarations] std::not1(std::ptr_fun<int, int>(::isspace)))); ^ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/functional:1100:1: note: 'ptr_fun<int, int>' has been explicitly marked deprecated here _LIBCPP_DEPRECATED_IN_CXX11 inline _LIBCPP_INLINE_VISIBILITY ^ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:1054:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX11' # define _LIBCPP_DEPRECATED_IN_CXX11 _LIBCPP_DEPRECATED ^ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED' # define _LIBCPP_DEPRECATED attribute ((deprecated)) ^ In file included from ../Libraries/acdriver/Sources/Compile/AppIconSet.cpp:9: In file included from ../Libraries/acdriver/Headers/acdriver/Compile/AppIconSet.h:12: In file included from ../Libraries/xcassets/Headers/xcassets/Asset/Asset.h:15: ../Libraries/libutil/Headers/libutil/Base.h:32:32: error: 'ptr_fun<int, int>' is deprecated [-Werror,-Wdeprecated-declarations] std::not1(std::ptr_fun<int, int>(::isspace))).base(), ^ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/functional:1100:1: note: 'ptr_fun<int, int>' has been explicitly marked deprecated here _LIBCPP_DEPRECATED_IN_CXX11 inline _LIBCPP_INLINE_VISIBILITY ^ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:1054:39: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX11' # define _LIBCPP_DEPRECATED_IN_CXX11 _LIBCPP_DEPRECATED ^ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:1043:48: note: expanded from macro '_LIBCPP_DEPRECATED' # 318 define _LIBCPP_DEPRECATED attribute ((deprecated)) ^ 2 errors generated. ninja: build stopped: subcommand failed. make: *** [all] Error 1

$ cmake --version cmake version 3.17.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).

$ ninja --version 1.10.0

I'm trying to run xcbuild on a project, and I've hit a wall where actool is failing without printing any kind of diagnostic as to why. Here's what I see:

CompileAssetCatalog ./MacVim-cllqusjqgekwejbmyolphrbvwgvq/Build/Products/Release/PSMTabBarControl.bundle/Contents/Resources /private/var/folders/cs/l6c6g06x0974_0w5pdg8dps80000gn/T/nix-build-macvim-8.2.539.drv-0/source/src/MacVim/PSMTabBarControl/Media.xcassets
    cd /private/var/folders/cs/l6c6g06x0974_0w5pdg8dps80000gn/T/nix-build-macvim-8.2.539.drv-0/source/src/MacVim/PSMTabBarControl
    export DEVELOPER_DIR=/nix/store/a99x032swmw9aghn8y4lvhjdimdm7z51-xcodebuild-0.1.2-pre
    export PATH=[long path]
    /nix/store/a99x032swmw9aghn8y4lvhjdimdm7z51-xcodebuild-0.1.2-pre/usr/bin/actool --product-type com.apple.product-type.framework --output-format human-readable-text --notices --warnings --export-dependency-info ./MacVim-cllqusjqgekwejbmyolphrbvwgvq/Build/Intermediates/PSMTabBarControl.build/Release/PSMTabBarControlFramework.build/assetcatalog_dependencies --output-partial-info-plist ./MacVim-cllqusjqgekwejbmyolphrbvwgvq/Build/Intermediates/PSMTabBarControl.build/Release/PSMTabBarControlFramework.build/assetcatalog_generated_info.plist --target-device mac --sticker-pack-identifier-prefix com.positivespinmedia.PSMTabBarControlFramework.sticker-pack. --compile ./MacVim-cllqusjqgekwejbmyolphrbvwgvq/Build/Products/Release/PSMTabBarControl.bundle/Contents/Resources /private/var/folders/cs/l6c6g06x0974_0w5pdg8dps80000gn/T/nix-build-macvim-8.2.539.drv-0/source/src/MacVim/PSMTabBarControl/Media.xcassets --platform macosx --minimum-deployment-target 10.12
/* com.apple.actool.errors */
: error: unable to create compiled asset writer

/* com.apple.actool.warnings */
: warning: product type not supported
: warning: platform not supported
: warning: sticker pack not supported
: warning: target device not supported

The warnings seem harmless; it's targeting a mac framework. "sticker pack" is just because of the --sticker-pack-identifier-prefix flag, it's not actually building a sticker pack.

But then, it just fails, saying "unable to create compiled asset writer". No explanation as to why not.

As best I can figure, that means CreateWriter failed, but there's no info as to why it failed or what I can do about this to move forward.

The CopyPNGFile rule invokes a tool copypng to do the actual copying. Xcode's version of copypng here is a Perl script that optionally invokes pngcrush. I expected xcbuild to have something similar, but xcbuild doesn't seem to bundle a copypng at all, so the CopyPNGFile rule simply fails.