Innovating Watercolour Art With AI: The Role Of TRL And OpenEnv

🔍 Read the full analysis: Innovating Watercolour Art With AI: The Role Of TRL And OpenEnv on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

An independent engineer has released a complete, open-source reproduction of Surya Narreddi’s viral watercolour AI model, built with TRL and OpenEnv on Hugging Face infrastructure. This project includes the training pipeline, datasets, reference pool, and trained models, enabling transparent and replicable research into AI aesthetics and reinforcement learning.

The reproduction closely follows Narreddi’s original concept, where a language model generates JavaScript code that creates watercolour paintings through the p5.js library. The model is trained via reinforcement learning against a composite reward function that emphasizes style, code length, and adherence to aesthetic preferences, with the latter guided by a vision model called Qwen3-VL-30B-A3B-Instruct. All artifacts—including the datasets, training scripts, and models—are openly hosted on Hugging Face, marking a significant step toward transparency in AI art research.

The project tests whether reinforcement learning can optimize a model for aesthetic taste rather than correctness, a departure from typical RL applications that focus on verifiable answers. The training involves 110 steps, 240 episodes per step, and uses LoRA to fine-tune a Qwen/Qwen3.5-35B model, with the entire pipeline initiated through a single command. The author emphasizes that this open release removes barriers for further experimentation and validation in AI art and aesthetic modeling.

Implications for AI Art and Transparent Research

This project demonstrates how reinforcement learning can be applied to optimize AI models based on aesthetic preferences, opening new avenues for AI-generated art that prioritizes style over correctness. The open release of datasets, code, and trained models fosters transparency and collaboration, enabling researchers and artists to build upon this work. It also provides a practical example of how open-source tools like TRL and OpenEnv can facilitate complex AI training pipelines, potentially accelerating innovation in AI art and creative coding.

Historical and Technical Background of AI Watercolour Projects

The project situates itself within a lineage of early generative AI art, from DeepDream (2015) to neural-network portraits by Mario Klingemann, and curated datasets like Anna Ridler’s tulip project. Surya Narreddi’s initial viral video in August, which showcased AI-generated watercolours, sparked widespread interest and highlighted the potential for AI to produce imperfect, handmade-looking art that contrasts with mainstream image models’ often polished outputs. The original project trained a language model to generate JavaScript code that creates watercolour effects, emphasizing interpretability and editability of each brushstroke. The open reproduction extends this by releasing all artifacts, thus enabling community validation and further experimentation.

“This open implementation allows anyone to explore how reinforcement learning can shape aesthetic preferences in AI-generated art.”

— Thorsten Meyer, project creator

Open Questions About Model Quality and Aesthetic Optimization

It remains unclear how closely the reproduced outputs match the original viral watercolour paintings in terms of quality and style. The project does not yet provide quantitative comparisons or final verdicts on which reward mix performs best. The full technical report from Narreddi, which is expected to clarify some of these points, has not yet been published. Additionally, how well the open implementation generalizes to other artistic styles or more complex compositions is still uncertain.

Future Developments and Community Engagement

The next steps include the release of Narreddi’s full technical report, which should provide deeper insights into the methodology and results. The open-source artifacts invite community validation, replication, and extension, potentially leading to new AI art tools that optimize for aesthetic taste. Researchers and artists are expected to experiment with different reward configurations, datasets, and model architectures, advancing the understanding of how reinforcement learning can influence creative AI outputs.

Key Questions

What is the main innovation of this open reproduction?

The project provides a fully open pipeline for training and evaluating AI models that generate watercolour art using reinforcement learning based on aesthetic preferences, with all datasets, scripts, and models openly available.

How does this work differ from typical AI art models?

Unlike standard generative models that optimize for pixel-level accuracy or style transfer, this project trains the model to prefer aesthetic qualities through reinforcement learning guided by human-like judgment models.

Can I use these artifacts to create my own watercolour AI art?

Yes, all datasets, scripts, and trained models are openly released on Hugging Face, allowing anyone to replicate or adapt the pipeline for their own projects.

What are the limitations of this project?

The quality and style fidelity of the generated paintings relative to the original viral videos are not yet quantitatively assessed. The full technical details and evaluation results are pending publication.

How might this influence future AI art research?

This work demonstrates the feasibility of optimizing AI models for aesthetic taste, encouraging more transparent, community-driven exploration of creative AI systems.

Primary source: Hugging Face · via ThorstenMeyerAI.com