🍯 Awesome log aggregator for iOS

Related tags

Logging Puree-Swift
Overview

Puree

Build Status Language Carthage compatible CocoaPods Compatible Platform License

Description

Puree is a log aggregator which provides the following features.

  • Filtering: Log entries can be processed before being sent. You can add common parameters, do random sampling, ...
  • Buffering: Log entries are stored in a buffer until it's time to send them.
  • Batching: Multiple log entries are grouped and sent in one request.
  • Retrying: Automatically retry to send after some backoff time if a transmission error occurred.

Puree helps you unify your logging infrastructure.

Currently in development so the interface might change.

Installation

Carthage

github "cookpad/Puree-Swift"

CocoaPods

use_frameworks!

pod 'Puree', '~> 5.0'

Swift PM

The Swift Package Manager is a tool for automating the distribution of Swift code and is integrated into the swift compiler. It is in early development, but Puree-Swift does support its use on supported platforms.

Once you have your Swift package set up, adding Puree-Swift as a dependency is as easy as adding it to the dependencies value of your Package.swift.

dependencies: [
    .package(url: "https://github.com/cookpad/Puree-Swift.git", .upToNextMinor(from: "5.2.0"))
]

Usage

Define your own Filter/Output

Filter

A Filter should convert any objects into LogEntry.

import Foundation
import Puree

struct PVLogFilter: Filter {
    let tagPattern: TagPattern

    init(tagPattern: TagPattern) {
        self.tagPattern = tagPattern
    }

    func convertToLogs(_ payload: [String: Any]?, tag: String, captured: String?, logger: Logger) -> Set {
        let currentDate = logger.currentDate

        let userData: Data?
        if let payload = payload {
            userData = try! JSONSerialization.data(withJSONObject: payload)
        } else {
            userData = nil
        }
        let log = LogEntry(tag: tag,
                           date: currentDate,
                           userData: userData)
        return [log]
    }
}

Output

An Output should emit log entries to wherever they need.

The following ConsoleOutput will output logs to the standard output.

class ConsoleOutput: Output {
    let tagPattern: TagPattern

    required init(logStore: LogStore, tagPattern: TagPattern) {
        self.tagPattern = tagPattern
    }

    func emit(log: LogEntry) {
        if let userData = log.userData {
            let jsonObject = try! JSONSerialization.jsonObject(with: userData)
            print(jsonObject)
        }
    }
}
BufferedOutput

If you use BufferedOutput instead of raw Output, log entries are buffered and emitted on a routine schedule.

class LogServerOutput: BufferedOutput {
    override func write(_ chunk: BufferedOutput.Chunk, completion: @escaping (Bool) -> Void) {
        let payload = chunk.logs.flatMap { log in
            if let userData = log.userData {
                return try? JSONSerialization.jsonObject(with: userData, options: [])
            }
            return nil
        }
        if let data = try? JSONSerialization.data(withJSONObject: payload, options: []) {
            let task = URLSession.shared.uploadTask(with: request, from: data)
            task.resume()
        }
    }
}

Make logger and post log

After implementing filters and outputs, you can configure the routing with Logger.Configuration.

import Puree

let configuration = Logger.Configuration(filterSettings: [
                                             FilterSetting {
                                                 PVLogFilter(tagPattern: TagPattern(string: "pv.**")!)
                                             }
                                         ],
                                         outputSettings: [
                                             OutputSetting {
                                                 PVLogOutput(logStore: $0, tagPattern: TagPattern(string: "activity.**")!)
                                             },
                                             OutputSetting {
                                                 ConsoleOutput(logStore: $0, tagPattern: TagPattern(string: "pv.**")!)
                                             },
                                             OutputSetting {
                                                 LogServerOutput(logStore: $0, tagPattern: TagPattern(string: "pv.**")!)
                                             },
                                         ])
let logger = try! Logger(configuration: configuration)
logger.postLog(["page_name": "top", "user_id": 100], tag: "pv.top")

Using this configuration, the expected result is as follows:

tag name -> [ Filter Plugin ] -> [ Output Plugin ]
pv.recipe.list -> [ PVLogFilter ] -> [ ConsoleOutput ], [ LogServerOutput ]
pv.recipe.detail -> [ PVLogFilter ] -> [ ConsoleOutput ], [ LogServerOutput ]
activity.recipe.tap -> ( no filter ) -> [ ConsoleOutput ]
event.special -> ( no filter ) -> ( no output )

We recommend suspending loggers while the application is in the background.

class AppDelegate: UIApplicationDelegate {
    func applicationDidEnterBackground(_ application: UIApplication) {
        logger.suspend()
    }

    func applicationWillEnterForeground(_ application: UIApplication) {
        logger.resume()
    }
}

Tag system

Tag

A tag is consisted of multiple term delimited by .. For example activity.recipe.view, pv.recipe_detail. You can choose your tags logged freely.

Pattern

Filter, Output and BufferedOutput plugins are applied to log entries with a matching tag. You can specify tag pattern for plugin reaction rules.

Simple pattern

Pattern aaa.bbb matches tag aaa.bbb, doesn't match tag aaa.ccc (Perfect matching).

Wildcard

Pattern aaa.* matches tags aaa.bbb and aaa.ccc, but not aaa or aaa.bbb.ccc (single term).

Pattern aaa.** matches tags aaa, aaa.bbb and aaa.bbb.ccc, but not xxx.yyy.zzz (zero or more terms).

Log Store

In the case an application couldn't send log entries (e.g. network connection unavailable), Puree stores the unsent entries.

By default, Puree stores them in local files in the Library/Caches directory.

You can also define your own custom log store backed by any storage (e.g. Core Data, Realm, YapDatabase, etc.).

See the LogStore protocol for more details.

License

Please do read the License before contributing.

Comments
  • Simplify Output initializer interface

    Simplify Output initializer interface

    Motivation

    I just want to pass dependencies to Output class which inherits BufferedOutput. The interface of initializer can't be changed because it's required by Output protocol. But the initializer is used only in OutputSetting and no need to instantiate from common initializer instead of closure.

    What changed

    I changed to use closure to instantiate OutputSetting and remove OutputOptions because no need to use common interface of initializer. And I added InstantiatableOutput for backward compatibility. If you used OutputSetting.init(O.Type,tagPattern: TagPattern), need to conform InstantiatableOutput

    opened by kateinoigakukun 5
  • BufferedOutput timer does not schedule when setup from a background queue

    BufferedOutput timer does not schedule when setup from a background queue

    As of #16, the Logger starts all outputs on a background queue. As a result of this, the BufferedOutput.setUpTimer() function is now being executed on the background queue set by the logger and not the main queue as it was in 3.3.0.

    This PR currently introduces a unit test that replicates the issue.

    It appears that this can be simply fixed by scheduling the timer on RunLoop.main instead of RunLoop.current however I'm not familiar with Puree enough to know if this change will be safe or not so I have the following questions:

    1. Is there a better way to write this unit test? I worry that it might be unreliable.
    2. Does scheduling the Timer to run on RunLoop.main defeat the object of #16? I think that it is ok to do this but just want to be sure that the results (BufferedOutput.tick(_:) will run on the main thread) are acceptable.

    I can commit a fix once it's been decided how we should proceed

    opened by liamnichols 5
  • [Proposal] Don't allow empty, space or newline only string in tag pattern.

    [Proposal] Don't allow empty, space or newline only string in tag pattern.

    Hi✋ This is tiny proposal. I think that tag pattern shouldn't allow empty, space or newline only string.

    In validation, returns false asap if found string that empty, space or newline only in the tag pattern.

    opened by ra1028 4
  • Add public memberwise initializer

    Add public memberwise initializer

    Motivation

    I wanted to specify a custom configuration forBufferdOutput, but when I initialize the configuration, I got the following error. initializer is inaccessible due to 'internal' protection level.

    What changed

    • I've added a public memberwise initializer according to the following document. 7ab39af

    https://docs.swift.org/swift-book/LanguageGuide/AccessControl.html

    As with the default initializer above, if you want a public structure type to be initializable with a memberwise initializer when used in another module, You must provide a public memberwise initializer yourself as part of the type's definition.

    • Changed to use the sizeLimit private property like any other properties. ae8aa95
    opened by nkmrh 3
  • Add data size limit to BufferedOutput

    Add data size limit to BufferedOutput

    • add chunkDataSizeLimit to BufferedOutput.Configuration
    • send logs before total data size in buffer reached its size limit even if buffered log count is less than count limit
    • add a test case
    opened by hiragram 3
  • Crash when deallocating Logger

    Crash when deallocating Logger

    Apparently (after Googling), Swift classes are deallocated from the thread that they were last retained on meaning that you cannot guarantee the thread that deinit is executed from.

    In the case of Logger, if you last called a function where self was captured internally in it's internal dispatch queue (i.e Logger.resume()) then the deinit function is executed from the internal dispatch queue.

    In the current implementation, deinit calls Logger.shutdown() that then attempts to execute dispatchQueue.sync { ... } while already on the internal dispatch queue resulting in the framework crashing with EXC_BAD_INSTRUCTION

    screenshot 2019-01-14 at 16 33 56

    You can reproduce the scenario with the following test case:

    func testLoggerDeallocation() {
    
        let configuration = Logger.Configuration(logStore: logStore,
                                                 dateProvider: DefaultDateProvider(),
                                                 filterSettings: [FilterSetting(PVLogFilter.self, tagPattern: TagPattern(string: "pv")!)],
                                                 outputSettings: [OutputSetting(PVLogOutput.self, tagPattern: TagPattern(string: "pv")!)])
        var logger: Logger? = try? Logger(configuration: configuration)
        XCTAssertNotNil(logger)
    
        logger?.postLog(["foo":"bar"], tag: "pv")
        logger?.suspend()
    
        weak var weakLogger: Logger? = logger
        logger?.resume()
        logger = nil
    
        let exp = expectation(description: #function)
        DispatchQueue.main.asyncAfter(deadline: .now() + 1) {
            XCTAssertNil(weakLogger)
            exp.fulfill()
        }
        waitForExpectations(timeout: 10.0, handler: nil)
    }
    

    This change removes the deinit implementation as it seems unnecessary although I'm not sure if we actually need to be able to suspend the outputs or not still?

    If we still need to suspend the outputs then maybe I should add it back but call dispatchQueue.async instead of .sync?

    opened by liamnichols 3
  • Support AppExtension

    Support AppExtension

    Thank you for awesome library.

    Motivation and Context

    We are want to use an AppExtension project. (ex: Share Extension, Today Extension)

    Description

    If this option set to YES. Puree-Swift can't use NS_EXTENSION_UNAVAILABLE apis.

    • UIApplication.shared
    opened by shogo4405 2
  • Small enhancement to  README.md

    Small enhancement to README.md

    Minor Enhancement

    • Was going through your repo, saw a small enhancement, which I guess , it's quiet important too, so I made the changes.

    Changes

    • Added the License section in Readme.md just to enhance the feel of readme.

    Future Updates

    • Right now going through your repo, trying my best to add a feature, soon I will push a PR :).

    Happy Coding!

    opened by gokulnair2001 1
  • Throwable LogStore initializer

    Throwable LogStore initializer

    Motivation and Context

    Currently, FileLogStore has an IUO property. In the real world apps, it sometimes causes crashes.

    So I modified the initialization interface of LogStore to avoid IUO properties.

    Description

    I use a throwable initializer instead of prepare method.

    Compatibility

    This PR doesn't impact users who uses default LogStores. Most users may use default LogStore. Users who own their custom LogStores have to change their implements to adapt to the new interface.

    opened by giginet 1
  • Adds different retry modes.

    Adds different retry modes.

    Be explicit about retry strategy when a write returns.

    An use case would be when the network becomes unavailable. We should suspend retries and the output until the next time we resume the output (when the network resumes)

    Intended behavior and usage:

    When a chunk write fails due to unavailable internet connection, we will stop retrying. We will also drop the failed chunk from currentWritingChunks, allowing it to be re-flushed after we resume the output. We will make the following changes in the calling app:

    1. Suspend the output on app backgrounding: there is no need to continue the flush timer.
    2. Resume the output after network becomes available, AND after app foregrounding. The reason it needs to be an AND here is that if the network becomes available when the app is backgrounded, we’re not sure we’ll receive the notification. However, it’s okay to resume multiple times because currentWritingChunks are not re-added to the buffer to be flushed.
    3. Set retry count and intervals configurations to be shorter than they are now. Due to the changed mentioned, we reduce some unnecessary consumption of a chunk's retry limit. However, we cannot make these settings too low, because the connection could simply be bad for a long time without the phone being in airplane mode.

    Effects:

    • We will not attempt to retry a chunk when the phone is in airplane mode or explicitly disconnected from the carrier.
    • If a disconnection is not detected, queued retries will be unaffected after app backgrounding (but will not execute while in background)
    • The downside is that, once the app returns to the foreground, the dropped chunks will immediately be re-created and retried, whereas previously they would wait until their next retry interval. If this causes a network crowding issue, we can explore staggering the requests.

    App restart:

    • Previously, chunks that exceeded their retry count will be attempted again after app restart (force quit). This will continue to be the case, with the addition that dropped chunks will also be retried.
    • Either way, this is dangerous because invalid entries will never clear out of the system if there is a programmer error. We should add a lifetime limit for each log entry, either by timestamp or by load count.
    opened by plimc 1
  • 4.0.0

    4.0.0

    This PR merges 4.0.0-beta branch into master.

    I added this commit to bump up version. https://github.com/cookpad/Puree-Swift/commit/4e0a20b53aed74cb7dc0be04ec4774d36d8be5c5

    opened by giginet 1
Releases(5.2.0)
Owner
Cookpad
Cookpad
📱💬🚦 TinyConsole is a micro-console that can help you log and display information inside an iOS application, where having a connection to a development computer is not possible.

TinyConsole TinyConsole is a tiny log console to display information while using your iOS app and written in Swift. Usage Wrap your Main ViewControlle

Devran Cosmo Uenal 2k Jan 3, 2023
Customizable Console UI overlay with debug log on top of your iOS App

AEConsole Customizable Console UI overlay with debug log on top of your iOS App AEConsole is built on top of AELog, so you should probably see that fi

Marko Tadić 142 Dec 21, 2022
Gedatsu provide readable format about AutoLayout error console log

Gedatsu Gedatsu provide readable format about AutoLayout error console log Abstract At runtime Gedatsu hooks console log and formats it to human reada

bannzai 520 Jan 6, 2023
Gedatsu provide readable format about AutoLayout error console log

Gedatsu Gedatsu provide readable format about AutoLayout error console log Abstract At runtime Gedatsu hooks console log and formats it to human reada

bannzai 475 Jun 24, 2021
Puree is a log collector which provides some features like below

Puree Description Puree is a log collector which provides some features like below Filtering: Enable to interrupt process before sending log. You can

Cookpad 149 Oct 18, 2022
A Swift-based API for reading from & writing to the Apple System Log (more commonly known somewhat inaccurately as "the console")

CleanroomASL Notice: CleanroomASL is no longer supported The Apple System Log facility has been deprecated by Apple. As a result, we've deprecated Cle

Gilt Tech 62 Jan 29, 2022
Log every incoming notification to view them again later, also includes attachments and advanced settings to configure

Vē Natively integrated notification logger Installation Add this repository to your package manager

alexandra 43 Dec 25, 2022
A custom logger implementation and Task Local helper for swift-log

LGNLog A custom logger implementation and TaskLocal helper for Swift-Log. Why and how This package provides two and a half things (and a small bonus):

17:11 Games 0 Oct 26, 2021
Automate box any value! Print log without any format control symbol! Change debug habit thoroughly!

LxDBAnything Automate box any value! Print log without any format control symbol! Change debug habit thoroughly! Installation You only need drag LxD

DeveloperLx 433 Sep 7, 2022
A log should tell a story, not drown the reader in irrelevance.

StoryTeller A log should tell a story, not drown the reader in irrelevance Story Teller is an advanced logging framework that takes an entirely differ

Derek Clarkson 10 Sep 20, 2019
A logging backend for swift-log that sends logging messages to Logstash (eg. the ELK stack)

LoggingELK LoggingELK is a logging backend library for Apple's swift-log The LoggingELK library provides a logging backend for Apple's apple/swift-log

null 17 Nov 15, 2022
Log messages to text files and share them by email or other way.

LogToFiles How to log messages to text files and share them by email or share center. 1 - Add the Log.swift file to your App 2 - Just log the messages

Miguel Chaves 0 Jan 9, 2022
Simply, Logify provides instant colorful logs to improve log tracking and bug tracing

Logify Simply, Logify provides instant colorful logs to improve log tracking and bug tracing. Why I need to use Logify? As discussed before in a lot o

Furkan KAPLAN 13 Dec 28, 2022
100 Days of SwiftUI log

100DaysOfSwiftUI 100 Days of SwiftUI log Table of Contents Day 1: variables, constants, strings, and numbers Day 1 Variables store values and can be c

Maegan Wilson 3 Oct 18, 2022
Simple Design for Swift bridge with Javascript. Also can get javascript console.log.

SDBridgeOC is here. If your h5 partner confused about how to deal with iOS and Android. This Demo maybe help. YouTube video is here. bilibili Video is

null 20 Dec 28, 2022
Shows your current framerate (fps) in the status bar of your iOS app

WatchdogInspector Shows your current framerate (fps) in the status bar of your iOS app Be a good citizen! Don't block your main thread! WatchdogInspec

Christian Menschel 510 Nov 24, 2022
TraceLog is a highly configurable, flexible, portable, and simple to use debug logging system for Swift and Objective-C applications running on Linux, macOS, iOS, watchOS, and tvOS.

Please star this github repository to stay up to date. TraceLog Introduction TraceLog is a highly configurable, flexible, portable, and simple to use

Tony Stone 52 Oct 28, 2022
In-App iOS Debugging Tool With Enhanced Logging, Networking Info, Crash reporting And More.

The debugger tool for iOS developer. Display logs, network request, device informations, crash logs while using the app. Easy accessible with its bubble head button ?? . Easy to integrate in any apps, to handle development or testing apps easier. First version, there is plenty of room for improvement.

Remi ROBERT 1.8k Dec 29, 2022
Bugfender SDK for iOS, a remote logger tailor-made for mobile

Bugfender SDK for iOS Bugfender is a cloud service to collect mobile application logs. Developers can control log sending programmatically and manuall

Bugfender 69 Dec 4, 2022