2.5.4

What's new?

• Add support for 3 new vision architectures (Swin, DeiT, Yolos) in https://github.com/xenova/transformers.js/pull/262. Check out the Hugging Face Hub to see which models you can use!
  • Swin for image classification. e.g.:

    let url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/tiger.jpg';
    let classifier = await pipeline('image-classification', 'Xenova/swin-base-patch4-window7-224-in22k');
    let output = await classifier(url, { topk: null });
    // [
    //   { label: 'Bengal_tiger', score: 0.2258443683385849 },
    //   { label: 'tiger, Panthera_tigris', score: 0.21161635220050812 },
    //   { label: 'predator, predatory_animal', score: 0.09135803580284119 },
    //   { label: 'tigress', score: 0.08038495481014252 },
    //   // ... 21838 more items
    // ]

  • DeiT for image classification. e.g.,:

    let url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/tiger.jpg';
    let classifier = await pipeline('image-classification', 'Xenova/deit-tiny-distilled-patch16-224');
    let output = await classifier(url);
    // [{ label: 'tiger, Panthera tigris', score: 0.9804046154022217 }]

  • Yolos for object detection. e.g.,:

    let url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/cats.jpg';
    let detector = await pipeline('object-detection', 'Xenova/yolos-small-300');
    let output = await detector(url);
    // [
    //   { label: 'remote', score: 0.9837935566902161, box: { xmin: 331, ymin: 80, xmax: 367, ymax: 192 } },
    //   { label: 'cat', score: 0.94994056224823, box: { xmin: 8, ymin: 57, xmax: 316, ymax: 470 } },
    //   { label: 'couch', score: 0.9843178987503052, box: { xmin: 0, ymin: 0, xmax: 639, ymax: 474 } },
    //   { label: 'remote', score: 0.9704685211181641, box: { xmin: 39, ymin: 71, xmax: 179, ymax: 114 } },
    //   { label: 'cat', score: 0.9921762943267822, box: { xmin: 339, ymin: 17, xmax: 642, ymax: 380 } }
    // ]

• Documentation improvements by @perborgen in https://github.com/xenova/transformers.js/pull/261

New contributors 🤗

• @perborgen made their first contribution in https://github.com/xenova/transformers.js/pull/261

Full Changelog: https://github.com/xenova/transformers.js/compare/2.5.3...2.5.4