It seems that some of the regexps in the PHP library have been generated by hand. This PR proposes to generate them automatically using the s9e\RegexpBuilder library.

At this time, this PR should be considered a proof-of-concept. I don't know what list of emoji sequences the regexp is supposed to match so the current regexp in this PR is based on the list of fully-qualified sequences from emoji.json plus the U+1F700..U+1F800 range and the U+1FA70..U+1FB00 range that appear to be matched by the original regexp.

If you'd like to merge this change, let me know what sequences should be matched and any other files that should be automatically patched and I'll update this PR accordingly.

To run the patching script, you'll need a modern version of PHP (>= 7.1 I think?) with the Intl extension enabled. Run the following at the root of the project:

composer up
php lib/php/patchFiles.php 

We have page with many user inputs

With 544 inputs and use toImage in 3.1.7, it takes ~1.55s

After we upgrade to 4.0.0.. times are not usable With 544 input and use toImage in 4.0.0, it takes ~20.8s

I did some demos (check console for times): v3.1.7 - https://jsfiddle.net/76qcth83/ ~0.07s v4.0.0 - https://jsfiddle.net/mzhqyagv/ ~9s v4.5.0 - https://jsfiddle.net/6dba573x/ ~9s

Edit: I tested v5.0.0 and performance is even worse https://jsfiddle.net/axsy9b2h/1/ ~15s

So you moved from 0.07s to 15s. Library is now unusable when you need to process more user inputs.

If you have ascii support enabled, then the <3 breaks conversion (at least for the JS library).

You can try it out on your demo page: https://demos.joypixels.com/latest/ascii-smileys.html

We have recently upgraded from emojiOne v4.5 to JoyPixels v5.0. We use the toShort function included in the joypixels.js file...

var shortcode = joypixels.toShort(emoji);

However, since upgrading from v4.5 to v5.0, we have noticed some eemoji shortcodes are outputting wrong. For example...

v4.5: slight_smile v5.0: slightly_smiling_face https://www.joypixels.com/emoji/slightly-smiling-face

v4.5: thumbsup v5.0: thumbup https://www.joypixels.com/emoji/thumbs-up

v4.5: thumbsdown v5.0: thumbdown https://www.joypixels.com/emoji/thumbs-down

All the links on the joypixels website suggest the shortcodes in v4.5 were correct.

"1f972": {
        "name": "smiling face with tear",
        "unicode_version": 13,
        "category": "people",
        "order": 16,
        "display": 1,
        "shortname": ":smiling_face_with_tear:",
        "shortname_alternates": [],
        "ascii": [],
        "humanform": 0,
        "diversity_base": 1,
        "diversity": null,
        "diversity_children": [],
        "gender": [],
        "gender_children": [],
        "code_points": {
            "base": "1f972",
            "fully_qualified": "1f972",
            "decimal": "",
            "diversity_parent": null,
            "gender_parent": null
        },
        "keywords": [
            "uc13"
        ]
    },

This emoji was added to emoji.json for Unicode 13. I believe it is incorrectly marked as a diversity_base.

Please do forgive my ignorance here, I'm not trying to have a dig or anything this is just a genuine question. I have observed that after a major JoyPixels update that the licenses go for sale and the joypixels/emoji-assets get updated almost straight away, but this repository tends to lag behind.

What is usually the reason for this and when might you envisage the toolkit being updated for version 6.

Many thanks!

Both toImage and shortnameToImage return a path that returns an S3 AccessDenied error.

(new \JoyPixels\JoyPixels())->getClient()->toImage(':smile:')

Returns:

<img class="joypixels" alt="&#x1f604;" title=":smile:" src="https://cdn.jsdelivr.net/joypixels/assets/5.5/png/32/1f604.png"/>

https://cdn.jsdelivr.net/joypixels/assets/5.5/png/32/1f604.png

I'm using the latest version (5.5.0)

because in emoji.json some "fully_qualified" isn't correct, but in emoji_strategy correct.

For example: "Hand with Fingers Splayed" it has unicode U+1F590 U+FE0F, in emoji_strategy.json it correct unicode "1f590-fe0f", but in emoji.json just "1f590"

Hi, Ruleset.php is outdated for 5.5.0 release. Is there any way to regenerate it?

https://github.com/joypixels/emoji-toolkit/blob/master/lib/php/src/Ruleset.php

Hi there,

The Unicode emoji for 'MOBILE PHONE' (U+1F4F1) has a shortname of "iPhone" in the EmojiOne / JoyPixels library.

Is there any reason for this renaming? It presents a bias towards Apple, as there is no Android or competitor counterpart. Additionally, this presents a problem for license holders integrating emojis in to competing platforms to Apple.

Might it be better to apply iPhone as a shortname, using a name like mobilephone which is closer to the Unicode source name?

Version: 5.0.5 PHP version: 7.3.9

Code:

$client = new \JoyPixels\Client($ruleset = new \JoyPixels\Ruleset());

$client->asciiToShortname('foo :) bar');

PHP Warning: array_flip(): Can only flip STRING and INTEGER values!

in https://github.com/joypixels/emoji-toolkit/blob/master/lib/php/src/Client.php#L333

Hello,I have a hybrid app built in Ionic. I noticed that when parsing emojis, the performance locks up the device, rendering it unusable for a period of 3 - 4 seconds. I am using in conjunction with ngx-markdown but I isolated the issue to this library.

Thank you!

In the 6.5 release the following emoji all have diversity_base: 1 but I don't believe they should. They as expected have no diversity_children.

face_exhaling face_in_clouds face_with_spiral_eyes mending_heart heart_on_fire

First of all, thank you for developing a good project. I succeeded in applying joypixels in react, and I want to share components.

EmojiAsset.tsx

import joypixels from 'emoji-toolkit'

const EmojiAsset = (props: { emoji: string }) => {
  const html = () => {
    const styles = [
      `width: 1.2em`,
      `height: 1.2em`,
      'top: -1px',
      'position: relative'
    ]
    const html: string = (joypixels.toImage(props.emoji) as string)
      .split(`<img class="joypixels"`)
      .join(`<img class="joypixels" style="${styles.join(';')}"`)

    return { __html: html }
  }

  return (
    <span
      className="emojiAsset"
      style={{ display: 'inline-block' }}
      dangerouslySetInnerHTML={html()}
    />
  )
}

export default EmojiAsset

index.tsx

const SomePage = () => {
  return (
    <EmojiAsset emoji="📦 Module" />
  )
}

export default SomePage

Example

In JoyPixels\Client, the $unicodeRegexp value has a mistake that causes it to match more often than needed. In the library itself, this is only used by JoyPixels\Client::toShort.

Specifically, the regex contains this portion:

...|[\x{1F1E0}-\x{1F1FF}]{2}||[\x{1F468}-\x{1F469}\x{1F9D0}-\x{1F9DF}][\x{1F3FA}-\x{1F3FF}]?\x{200D}?[\x{2640}\x{2642}\x{2695}\x{2696}\x{2708}]?\x{FE0F}?|...

What's notable is the || part, which causes the regex to match empty strings. This causes toShortCallback to be called with an empty match for each (non-emoji) character. This returns immediately, but this the match and callback overhead can be significant. With a single | this works as expected.

For example, here's a basic performance test:

$runCount = 100;
$str = str_repeat("testing \xF0\x9F\x98\x83 ", 1000);
$client = new \JoyPixels\Client();

$s = microtime(true);
for ($i = 0; $i < $runCount; $i++) {
	$client->toShort($str);
}
$e = microtime(true);

echo ($e - $s) / $runCount;

This gives me the following results:

Run type | Time per execution (s) | Total time (s) ---------- | ------------------------ | -------------- Unfixed | 0.018859 | 1.88592 Fixed | 0.000538 | 0.0538551

Though easily derivable, the difference in the total run time shows the size of the regression as we go from 1.88 seconds to run the test to 0.05 with a single character change!

This change first appeared in 5.5 (af05016636de451c0bca8a2910fe503efff350e2). I assume this regex is auto generated, so this will presumably keep coming back unless the tool that causes the issue is fixed. Otherwise, a PR to resolve this is generally simple.

Potentially related, JoyPixels::$unicodeRegexp does not appear to have this issue, though the regex differs from the one in JoyPixels\Client. It did actually get the || in the 5.5 release as well (you can see it in the same commit), but it disappeared from there in 6.0. I'm not clear on why these two regexes are not identical, but I might be missing something.

Hello we use the license of animations 3.5 and on the different GIF file sizes, especially on small sizes a white pixelized !

outline is present. I checked other transparent GIF animations on Google or Discord and this white outline is often not present unlike Joypixel.

How to proceed so as to no longer have its embarrassing white contours? (very annoying for dark mode)

In addition, on older animation versions (3.2), the white outline problem was less present and the files were less heavy. Revise your gif export method.

Thanks for your time 😇

Capture d’écran 2020-11-24 à 23 45 44