Erik is an headless browser based on WebKit. An headless browser allow to run functional tests, to access and manipulate webpages using javascript.

Overview

Erik

Join the chat at https://gitter.im/phimage/Erik License Platform Language Issues CocoapodCarthage compatible

Become a Patron! Buy me a coffee

Erik is a headless browser based on WebKit and HTML parser Kanna.

An headless browser allow to run functional tests, to access and manipulate webpages using javascript.

let browser = Erik.visit(url: url) { document, error in
    // browse HTML element, click, submit form and more
}

Navigation

Go to an url

Erik.visit(url: url) { object, error in
    if let e = error {

    } else if let doc = object {
        // HTML Inspection
    }
}

How to get current url?

if let url = Erik.currentURL {..}

For multiple browsing you can create an instance of headless browser and use same functions

let browser = Erik()
browser.visitURL...

HTML Inspection

Search for nodes by CSS selector

for link in doc.querySelectorAll("a, link") {
    print(link.text)
    print(link["href"])
}

Edit first input field with name "user"

if let input = doc.querySelectorAll("input[name=\"user\"]").first {
    input["value"] = "Eric"
}

Submit a form

if let form = doc.querySelector("form[id='search']") as? Form {
    form.submit()
}

Evaluate some JavaScript

let javaScriptSource = "console.log("test");"
Erik.evaluate(javaScript:javaScriptSource) { (obj, err) -> Void in
    if let error = err {
        switch error {
            case ErikError.javaScriptError(let message):
            print(message)
            default :
            print("\(error)")
        }
    }
    else if let capturedValue = obj {
        // do something according to result
    }
}

capturedValue is the content of JavaScript variable resultErik Affect this variable in your JavaScript code.

let javaScriptSource = "console.log('test'); var resultErik = 1 + 1;"

Warning about DOM change

⚠️ All action on Dom use JavaScript and do not modify the actual Document object and its children Element.

You must use currentContent to get a refreshed Document object

Get current content

Erik.currentContent { (obj, err) -> Void in
    if let error = err {
    }
    else if let document = obj {
       // HTML Inspection
    }
}

Using Future

As an optional feature, you can use Future/Promise ( Erik use frameworks BrightFutures & Result)

Example to submit a google search

let url = NSURL(string:"https://www.google.com")!
let value = "Erik The Phantom of Opera"
// visit
var future: Future<Document, NSError> = Erik.visitFuture(url: url)
// fill input field
future = future.flatMap { document -> Future<Document, NSError> in
    if let input = document.querySelector("input[name='q']") {
        input["value"] = value
    }
    if let form = document.querySelector("form[name=\"f\"]") as? Form {
        form.submit()
    }
    return Erik.currentContentFuture()
}
// finally get final result as success or error
future.onSuccess { document in
    // parse result
}
future.onFailure { error in
    print("\(error)")
}

Limitation

On iOS 9 and macOS 10.11, you need to ensure you use https://, because iOS 9 and macOS 10.11 do not like apps sending or receiving data insecurely. If this something you want to override, click here to read about App Transport Security.

Links

Setup

Using cocoapods

CocoaPods is a centralized dependency manager for Objective-C and Swift. Go here to learn more.

  1. Add the project to your Podfile.

    use_frameworks!
    
    pod 'Erik'
    // or specific target
    target :test do
       pod 'Erik'
    end
  2. Run pod install and open the .xcworkspace file to launch Xcode.

Optional Future

Add pod 'Erik/Future' to your Podfile and run pod install.

Using carthage

Carthage is a decentralized dependency manager for Objective-C and Swift.

  1. Add the project to your Cartfile.

    github "phimage/Erik"
    

Roadmap

  • (WIP) WKWebView screenshot (webkit view privates api?)

Why Erik?

A well known headless browser is named PhantomJS and a very well known browser is Opera.

As a tribute I use Erik, firstname of the title character from Gaston Leroux's novel Le Fantôme de l'Opéra best known to English speakers as The Phantom of Opera

My name is also Erik. So egotistical to call a project using its firstname isn't it.

My only justification is that I was playing Metal Gear Solid V and the creator Hideo Kojima name appears over 100 times in the game. Coincidentally the full name of the game is Metal Gear Solid V : The Phantom Pain.

License

The MIT License. See the LICENSE file for more information.

Comments
  • Threading with promises

    Threading with promises

    Hi

    I seem to be having issues with promises. Erik.visit() and Erik.visitFuture() take a very long time when embedded in other promises. And then they eventually fail to provide a document as I think the request is timed out.

    I've tried with both BrightFutures and PromiseKit.

    Is this possibly a threading issue?

    opened by matadan 15
  • Swift 3 support

    Swift 3 support

    Trying to compile my app with Swift 3 and I get two errors about unwrapping optionals. I've chosen to use the legacy swift compiler, but I think 2.3 (which is what I assume it's using) is also incompatible.

    opened by mattahorton 14
  • Not able to submit a form

    Not able to submit a form

    Hello :)

    Thank you for this amazing project. I'm trying to log in on a Central Authentication Service (i think drupal...) so I have to fill a form with my username and password and then submit the form. I tried to submit the form or click on the button submit (or even both) but erik appears to be stuck on the login page. I tried to connect with HtmlUnit (Java) and it worked, but I need to do it on an iOS App.. Could someone help me? Here are the snippets :

    SWIFT :

    url = URL(string:"http://edt.insa-cvl.fr")!
            let username = "username"
            let password = "password"
            // visit
            var future: Future<Document, NSError> = Erik.visitFuture(url: url)
            // fill input field
            future = future.flatMap { doc -> Future<Document, NSError> in
                if let inputUsername = doc.querySelector("input[name=\"username\"]") {
                    inputUsername["value"] = username
                }
                
                if let inputPassword = doc.querySelector("input[name=\"password\"]") {
                    inputPassword["value"] = password
                }
                
                
                
                if let form = doc.querySelector("form[id=\"fm1\"]") as? Form {
                    form.submit()
                }
                
                if let buttonSubmit = doc.querySelector("input[name=\"submit\"]") {
                    buttonSubmit.click()
                }
    
            
                return Erik.currentContentFuture()
            }
            // finally get final result as success or error
            future.onSuccess { doc in
                print(String(describing: doc))
            }
            future.onFailure { error in
                print("\(error)")
            }
    
    
    

    JAVA :

    try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_52)) {
    		    final HtmlPage page = webClient.getPage("http://edt.insa-cvl.fr");
    		   final HtmlTextInput usernameField = (HtmlTextInput) page.getElementById("username");
    		    final HtmlPasswordInput passwordField = (HtmlPasswordInput) page.getElementById("password");
    		    
    		    usernameField.setValueAttribute("username");
    		    passwordField.setValueAttribute("password");
    		    
    		    final HtmlSubmitInput buttonSubmit = page.getElementByName("submit");
    		    final HtmlPage page2 = buttonSubmit.click();
    		     
    		     System.out.println(page2.asXml());
    
    		}
    
    

    Thanks in advance!

    opened by arnaudmdr 7
  • UI API called on a background thread - iOS 11 - Xcode 9

    UI API called on a background thread - iOS 11 - Xcode 9

    Hi!

    Erik unfortunately does not work on iOS 11. With the new Main Thread Checker in XCode 9 it prints out the following in the console:

    • Main Thread Checker: UI API called on a background thread: -[WKWebView navigationDelegate]
    • Main Thread Checker: UI API called on a background thread: -[WKWebView evaluateJavaScript:completionHandler:]

    Best regards, Ziad

    opened by ziadtamim 7
  • Could not build Objective-C module 'Erik'

    Could not build Objective-C module 'Erik'

    I installed Erik using podfile and the installation is successful, but it gives me this error when I do "import Erik". Below is my podfile.

    platform :ios, '12.0'

    target 'MyTritonLink' do use_frameworks! pod 'Erik'

    target 'MyTritonLinkTests' do inherit! :search_paths # Pods for testing end

    target 'MyTritonLinkUITests' do inherit! :search_paths # Pods for testing end

    end

    opened by jeffreyyu0602 6
  • Can't install pod properly:

    Can't install pod properly: "No such module 'Erik'"

    Hello,

    I'm trying to use Erik.. First I tried pod try Erik and selected the first demo application, but this gave me the following error:

    [!] Unable to satisfy the following requirements:
    
    - `FileKit (from `https://github.com/nvzqz/FileKit.git`, branch `develop`)` required by `Podfile`
    

    Then I tried to install Erik into a new Xcode project (xcode 9, iOS 11). Here's the pod file I used:

    target 'ErikPlayAgain' do
      use_frameworks!
    
      pod 'Erik'
    end
    

    When opening the project I get 22 error, even without import Erik. All errors in Kanna.switf that look very similar to this: Let 'kDefaultXmlParseOption' is private and cannot be referenced from a default argument value. When import Erik to my ViewController, I see this: No such module 'Erik'.

    This is a very trivial task, but I still can't make it work. What am I missing?

    opened by Wiingaard 4
  • testSubmit() fails

    testSubmit() fails

    I cloned the repo and ran tests.

    testSubmit() fails. I suspect this is related to #24 and will be fixed with #24.

    Please let me know if there is anything I can do to help resolve these.

    bug 
    opened by matadan 4
  • Could not build Objective-C module 'libxml2'

    Could not build Objective-C module 'libxml2'

    I encountered following error trying to build Erik on an empty iOS project using CocoaPods.

    <module-includes>:1:9: note: in file included from <module-includes>:1:
    #import "libxml2-kanna.h"
            ^
    /Users/gbm/Developer/Study/TestApp/Pods/Kanna/Modules/libxml2-kanna.h:1:9: note: in file included from /Users/gbm/Developer/Study/TestApp/Pods/Kanna/Modules/libxml2-kanna.h:1:
    #import <libxml2/libxml/HTMLtree.h>
            ^
    /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator10.2.sdk/usr/include/libxml2/libxml/HTMLtree.h:15:10: error: 'libxml/xmlversion.h' file not found
    #include <libxml/xmlversion.h>
             ^
    <unknown>:0: error: could not build Objective-C module 'libxml2'
    

    After some searching, I fixed the problem by adding $(SDKROOT)/usr/include/libxml2 to Pod project -> Erik target -> Build Settings -> Header Search Path

    Podfile.lock

    PODS:
      - Erik (2.0.0):
        - Erik/Core (= 2.0.0)
        - Kanna
      - Erik/Core (2.0.0):
        - Kanna
      - Kanna (2.1.1)
    

    macOS Version 10.12.3 Xcode Version 8.2.1

    bug 
    opened by gbmksquare 4
  • How does it work ?

    How does it work ?

    Sorry for this idiot question but, i dont know how it works.

    When i try to do :

    Erik.visitURL(NSURL(string: "https://google.com")!) { document, error in
                // browse HTML element, click, submit form and more
                print(document)
                print(error)
    }
    

    the output is empty markup html :

    Optional(<html><head></head><body></body></html>)
    nil
    

    I dont know if is it normal behaviour or anything else i have forgotten.

    opened by jchesne 4
  • Another Noob Question

    Another Noob Question

    I'm using XCode 11 and a package.swift file to configure my project. I'd rather keep using this, since it's what I'm familiar with.

    When adding phimage/Erik.git to the package file, I get the following error:

    https://github.com/phimage/Erik.git has no Package.swift manifest for version 1.1.1

    Is this my error, or something on the Erik side?

    opened by barrymcbride 3
  • Question: html -> pdf?

    Question: html -> pdf?

    Can Erik be used to generate a PDF file from the rendering of a webpage via either https and/or file url? (preferrably not as a image raster which is then placed into PDF)

    question 
    opened by marc-medley 3
  • Overriding navigationDelegate causes ErikError.noContent

    Overriding navigationDelegate causes ErikError.noContent

    Overriding navigationDeledate causes ErikError.noContent, if you remove the view.navigationDelegate = self line then it content is produced.

    class Test: LayoutEngineNavigationDelegate {
        let view: WKWebView
        let browser: Erik
        
        override init() {
            view = WKWebView()
            browser = Erik(webView: view)
            super.init()
            view.navigationDelegate = self
        }
        
        func test() {
            browser.visit(url: URL(string: "https://google.com")!) { (document, err) in
                debugPrint(err as Any)
                debugPrint(document as Any)
            }
        }
    }
    
    opened by L3afMe 1
  • in iOS, If I create multiple instances of Erik and each one loads a URL, it raises an ErikError.timeOutError

    in iOS, If I create multiple instances of Erik and each one loads a URL, it raises an ErikError.timeOutError

    Hello. Thanks for the great library.

    It seems that if I create multiple instances of Erik and each one loads a URL, it raises an ErikError.timeOutError. Running the following code on my iPad mini 4 raises an ErikError.timeOutError in most cases.

        let erikPool:[Erik] = [
            Erik(), Erik(), Erik(), Erik(), Erik(),
            Erik(), Erik(), Erik(), Erik(), Erik(),
        ]
        func testErik() {
            guard let url = URL(string: "https://www.google.com/") else { return }
            func startErik(id:String, url:URL, client:Erik) {
                client.visit(url: url) { (doc, err) in
                    if let err = err {
                        print("Erik error.", err.localizedDescription)
                        return
                    }
                    if let firstLine = doc?.innerHTML?.trimmingCharacters(in: .whitespacesAndNewlines).components(separatedBy: "\n").first {
                        print(id, "request done.", firstLine)
                    }else{
                        print(id, "request done. but no content?")
                    }
                }
            }
            for (i, erik) in erikPool.enumerated() {
                startErik(id: String(format: "test #%d", i), url: url, client: erik)
            }
        }
    

    Perhaps it's because the handleLoadRequestCompletion in Erik's WebKitLayoutEngine is busy looping, but it looks like the processing isn't up to speed. As a workaround to this issue, I added Thread.sleep(forTimeInterval: 0.1) to Erik's source code as shown below, and found that each request completed successfully.

        fileprivate func handleLoadRequestCompletion(completionHandler: (Error?) -> Void) {
            // wait load finish
            let condition = pageLoadedPolicy.continueCondition
            let max = Date().timeIntervalSince1970 + pageLoadTimeout
            while(condition(self)) {
                if pageLoadTimeout > 0 && Date().timeIntervalSince1970 > max  {
                    completionHandler(ErikError.timeOutError(time: pageLoadTimeout))
                    return
                }
            #if os(OSX)
                RunLoop.current.run(mode: RunLoop.Mode.default, before: Date.distantFuture)
            #else
                Thread.sleep(forTimeInterval: 0.1)
            #endif
            }
            completionHandler(nil)
        }
    

    I wasn't sure why I was using a loop to monitor this part of the code, so I decided to use the above workaround. However, you may be able to change this problem to something like a WKWebView.addObserver that monitors changes in "estimatedProgress" or "loading" without using a busy loop.

    opened by limura 0
  • Package Resolution Errors with Erik / Filekit

    Package Resolution Errors with Erik / Filekit

    Hey,

    Still trying to incorporate Erik into my project, and am running into package resolution errors on XCode 11.4. I'm running a Kitura/Postgresql/Swift-Kuery-ORM project and get the following message when trying to add Erik:

    because Erik >=5.1.0 depends on FileKit 6.0.0..<7.0.0 and Configuration >=3.0.0 depends on FileKit 0.0.1..<1.0.0, Erik is incompatible with Configuration. And because Swift-cfenv >=6.0.0 depends on Configuration 3.0.0..<4.0.0 and every version of Kitura-OpenAPI depends on Swift-cfenv 6.0.0..<7.0.0, Erik is incompatible with Kitura-OpenAPI. And because root depends on Kitura-OpenAPI 1.1.1..<2.0.0 and root depends on Erik 5.1.0..<5.2.0, version solving failed.

    Looks like a wide disparity between https://github.com/IBM-Swift/Configuration and Erik, but I'm still new at this and might be confused.

    opened by barrymcbride 5
  • Memory leak issue on iOS 12.1

    Memory leak issue on iOS 12.1

    I have a singleton to manage instance of Erik. And after complete task I release the instance but Erik still in memory. I use two Erik instances visit www.apple.com and found about 50MB memory leaked.

    My app has not refer the instance anymore. So I think it's could be leaking somewhere. Maybe due to WKScriptMessageHandler.

    Demo app: https://www.dropbox.com/s/luyes5cbz03jg0k/ErikDemo.zip?dl=0

    bug 
    opened by MainasuK 0
Releases(5.1.0)
  • 5.1.0(Oct 28, 2019)

  • 2.0.0(Oct 12, 2016)

  • 1.1.1(Sep 6, 2016)

    Fix #15 By default detect page loading end using the webkit navigationDeletage instead of loading boolean If you use your own instance of WKView in Erikinstance, and have already set a delegate, you can make it implement the protocol Navigable

    Other algorithm could be chosen to detect page loading (myErikInstance.layoutEngine as? WebKitLayoutEngine)?.pageLoadedPolicy =.loading // the loading boolean (myErikInstance.layoutEngine as? WebKitLayoutEngine)?.pageLoadedPolicy =.estimatedProgress // the estimatedProgress value (must be 1.0 except when starting 0.0)

    If no page loaded, NoContent is thrown when asking for currentContent An empty page is detected using pattern Erik.noContentPattern You can set to nil if you want to keep previous code state

    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Sep 6, 2016)

Owner
Eric Marchand
🧙🍺🥊🥋🏓
Eric Marchand
An NSURL extension for showing preview info of webpages

URLPreview An NSURL extension for showing preview info of webpages. You may want to use it if you want to mimick Facebook app's behavior when you post

Huong Do 198 Dec 5, 2022
An RSS, Atom and JSON Feed parser written in Swift

Features Atom RSS 0.90, 0.91, 1.00, 2.00 JSON Namespaces Dublin Core Syndication Content Media RSS iTunes Podcasting Tags Documentation Unit Test Cove

Nuno Dias 1k Jan 7, 2023
Erik is an headless browser based on WebKit. An headless browser allow to run functional tests, to access and manipulate webpages using javascript.

Erik Erik is a headless browser based on WebKit and HTML parser Kanna. An headless browser allow to run functional tests, to access and manipulate web

Eric Marchand 544 Dec 30, 2022
Mathias Köhnke 1.1k Dec 16, 2022
WebDomHandling - A Swift Package for handling JavaScript code between WebKit and Swift implemented by WebKit

WebDomHandling A Swift Package for handling JavaScript code between WebKit and S

null 0 Jan 23, 2022
JSPatch bridge Objective-C and Javascript using the Objective-C runtime. You can call any Objective-C class and method in JavaScript by just including a small engine. JSPatch is generally used to hotfix iOS App.

JSPatch 中文介绍 | 文档 | JSPatch平台 请大家不要自行接入 JSPatch,统一接入 JSPatch 平台,让热修复在一个安全和可控的环境下使用。原因详见 这里 JSPatch bridges Objective-C and JavaScript using the Object

bang 11.4k Jan 1, 2023
Simple Design for Swift bridge with Javascript. Also can get javascript console.log.

SDBridgeOC is here. If your h5 partner confused about how to deal with iOS and Android. This Demo maybe help. YouTube video is here. bilibili Video is

null 20 Dec 28, 2022
WebKit aims to provide platform agnostic isolated browser environments without the need for sketchy C bindings or a bloated V8 runtime.

WebKit WebKit aims to provide platform agnostic isolated browser environments without the need for sketchy C bindings or a bloated V8 runtime. Running

Linden 1 Nov 26, 2021
A functional tool-belt for Swift Language similar to Lo-Dash or Underscore.js in Javascript

Dollar Dollar is a Swift library that provides useful functional programming helper methods without extending any built in objects. It is similar to L

Ankur Patel 4.2k Jan 2, 2023
A functional tool-belt for Swift Language similar to Lo-Dash or Underscore.js in Javascript

Dollar Dollar is a Swift library that provides useful functional programming helper methods without extending any built in objects. It is similar to L

Ankur Patel 4.2k Jan 4, 2023
A functional tool-belt for Swift Language similar to Lo-Dash or Underscore.js in Javascript

Dollar Dollar is a Swift library that provides useful functional programming helper methods without extending any built in objects. It is similar to L

Ankur Patel 4.2k Jan 4, 2023
WKZombie is a Swift framework for iOS/OSX to navigate within websites and collect data without the need of User Interface or API, also known as Headless browser.

WKZombie WKZombie is an iOS/OSX web-browser without a graphical user interface. It was developed as an experiment in order to familiarize myself with

Mathias Köhnke 1.1k Dec 16, 2022
Utility to run the SPI-Server tests as a benchmark

spi-benchmark This package comprises a simple tool to run the SwiftPackageIndex-Server tests in a loop, logging the run times. The purpose is to colle

Swift Package Index 2 Mar 13, 2022
A repository that demonstrates the difficulty to run async tests with Xcode 13.2 beta on pre iOS-15 simulators

A repository that demonstrates the difficulty to run async tests with Xcode 13.2 beta on pre iOS-15 simulators This demonstration uses an iOS 13.7 sim

Gwendal Roué 0 Nov 1, 2021
Small library to easily run your tests directly within a Playground

[] (https://developer.apple.com/swift/) Build Status Branch Status master develop About PlaygroundTDD enables you to use TDD directly on Xcode Playgro

Whiskerz AB 317 Nov 22, 2022
PlaygroundTester enables you to easily run tests for your iPad Playgrounds 4 project.

PlaygroundTester PlaygroundTester is a package that enables you to add tests to your iPad Swift Playgrounds project. Installation Just add PlaygroundT

Paweł Łopusiński 36 Dec 13, 2022
An NSURL extension for showing preview info of webpages

URLPreview An NSURL extension for showing preview info of webpages. You may want to use it if you want to mimick Facebook app's behavior when you post

Huong Do 198 Dec 5, 2022
SwiftyOpenGraph - A swift library that retrieves structured open graph data from webpages.

SwiftyOpenGraph Usage Initialization Base Properties Types Installation License Usage Initialization You use SwiftyOpenGraph by initializing OpenGraph

Quintschaf 1 Jan 6, 2022
A simple project for using WebKit and SafariServices

SafariServices VS WebKit This respository is a simple project for using WebKit and SafariServices. WebKit is the layout and rendering engine behind Sa

Lee McCormick 0 Nov 9, 2021
This was built during my bootcamp using SwiftUI, WebKit and an API from Hacker News

HackerNewsReader This was built during my bootcamp using SwiftUI, WebKit and an API from Hacker News. This was programmed from scratch with SwiftUI. I

Wilson Woodyard 2 Feb 25, 2022