My 30-Image Experiment With Jev’s AI Vision

🔍 Read the full analysis: My 30-Image Experiment With Jev’s AI Vision on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

Thorsten Meyer reports that Jev 1.14 correctly identified the featured product in 9 of 10 product images in a small test, after TypeSafe enabled opt-in image input in its October 14 release. The experiment used 30 images from Meyer’s website and compared Jev 1.14 with 1.13, but the available report ends before presenting the results for news and essay images.

TypeSafe’s October 14 announcement introduced image input for Jev’s decision model and a new bounding-box response type in Jev 1.14. Image support also works with Jev 1.13, the company said. Both capabilities require an explicit request header, which TypeSafe described as a safeguard while the features remain too unreliable for it to recommend in production.

Meyer tested the models on the 30 most recent featured images on ThorstenMeyerAI.com: 10 product roundups, 15 news posts and five essays. For each image, he asked whether it fit its own headline, a randomly selected other headline and an unrelated plant-care headline. The product images also received a pick-one question asking which of 10 products appeared in the image. Meyer says the experiment involved about 60 requests across Jev 1.13 and 1.14.

For the 10 product images, Meyer reports that Jev 1.14 gave each image’s own headline a score from 0.65 to 0.89, with a median of 0.76. Random other headlines scored no higher than 0.18, with a median of 0.06; the unrelated plant headline scored no higher than 0.10. These are results from Meyer’s test, not a benchmark independently verified by TypeSafe.

What the Product Test Shows

The results suggest image input could support narrow, repeatable classification tasks, such as checking whether a product photo fits an article headline or selecting an item from a defined list. Those tasks fit Jev’s existing design: it returns structured answers, probabilities or scores rather than writing free-form descriptions.

The reported product results also show why confidence scores need careful handling. Jev 1.14 named the correct product in nine of 10 cases. Its error involved a mesh Wi-Fi image containing both a router with antennas and a cylindrical mesh unit; the model selected “Wi-Fi 7 router” with 0.60 confidence. Meyer says all correct answers had confidence of at least 0.77, but that pattern in a 10-image sample does not establish a dependable threshold for other images or tasks.

For publishers and developers, the release offers a way to test image-based decisions without changing Jev’s default behavior. The experiment’s limited sample and the missing results for two image categories mean it does not show how well the feature generalizes across sites, image styles or production workflows.

Jev’s Opt-In Image Release

Before this release, Jev accepted typed questions about text. Users provide a state and questions; the model returns answers such as yes-or-no probabilities, selections from a list or scores, each with a confidence value. TypeSafe described the model as a decision system rather than a text-writing assistant.

TypeSafe says images can be placed in the request state, question instructions or criteria for answer options. The release uses references in an asset map to insert an image wherever its reference appears in the request. The company says supported files are PNG or JPEG still images, 256 to 1,568 pixels per side, with a 16 MiB file limit and a 32 MiB request limit. The supplied article says images are billed as ordinary input tokens at Jev’s stated rate of $0.042 per million input tokens, with no separate image surcharge.

The bounding-box type is available only in Jev 1.14 and returns one box for one described target, together with a confidence score. TypeSafe also introduced an opt-in longer-context setting, but Meyer’s image experiment focused on the vision features. The company said it does not yet vouch for the new capabilities in production and advised users to evaluate them for their own needs.

“There is still way too much jaggedness for us to vouch for these for production use cases.”

— TypeSafe, in its Jev 1.14 release post, as quoted by Thorsten Meyer

Results Still Missing From the Test

The source text provided for this report stops during the section on news images, after the words “The news posts were a dif”. It gives no results for the 15 news images or the five essays, and no outcomes for the bounding-box feature. It is not possible to tell from the available material whether those findings were published later in the article.

The test also does not provide independently checked labels, a comparison with other vision models or enough cases to establish real-world accuracy. Meyer reports that Jev 1.13 identified all 10 products but assigned confidence below 0.5 to two; the article excerpt does not provide comparable headline-fit scores for that version. TypeSafe’s own warning about reliability remains relevant, and the results should be read as one site’s experiment rather than a general performance evaluation.

Further Testing Before Deployment

TypeSafe made Jev 1.14 available as jev-preview on October 14 and said the jev-latest alias would move to it about a week later, around October 21. The company also outlined deprecation periods for versions 1.13 and 1.14, while saying a formal policy was still to come.

For readers assessing the vision features, the next useful evidence would include the omitted news and essay results, repeated tests on a larger and independently labeled image set, and direct evaluation of bounding-box accuracy. TypeSafe has advised users to run their own evaluations and has not endorsed the new features for production use.

Key Questions

What changed in Jev 1.14?

TypeSafe added opt-in image input and a new bounding-box response type. The bounding-box feature is limited to Jev 1.14; image support also works with Jev 1.13, according to the release information cited by Meyer.

How many images did Meyer test?

Meyer says he tested 30 website images: 10 product roundup images, 15 news images and five essay images. The available source excerpt provides detailed results only for the 10 product images.

How did Jev 1.14 perform on product images?

In Meyer’s account, Jev 1.14 matched the product image to its own headline with scores from 0.65 to 0.89, and selected the right product in nine of 10 cases. These findings come from a small, single-site experiment.

Does TypeSafe recommend using the image features in production?

No. TypeSafe said the features still had too much reliability “jaggedness” for it to vouch for production use, and placed them behind explicit opt-in headers.

Are the news-image and bounding-box results available?

They are not included in the source material provided here. The excerpt cuts off during the news-image section and reports no bounding-box test outcome.

Source: ThorstenMeyerAI.com