# Ren, our first model. Public weights, public limits.

A decision model takes a situation and a list of options, then returns one option with a probability. To trust software to act alone, you need proof of when it's safe.

## Model details

Ren is trained once. It can work from option names alone and gets better with a few examples. It's English only.

- Parameters: 396M
- Training rows: 422,906
- Training sets: 38
- Compute: About $129
- Weights licence: CC BY-SA 4.0
- Version: v1.1.0

- Weights: https://huggingface.co/Secnd-AI/ren/tree/v1.1.0
- Code: https://github.com/Secnd-AI/ren

Built with Muse Spark Open. Ren is independently developed for Meta's platform and is not an official Meta product.

## Results

| Test | Ren | Comparison |
| --- | ---: | --- |
| Typed decisions, mixed test | 79.7% | OpenDecider-nano 79.6%, OpenDecider's run of Jev 1.13 75.4% |
| BANKING77, no examples | 69.0% | Jev 79.7%, from a published study |
| BANKING77, with examples | 89.9% with 8 | Jev 92.4% with 24, from the same study |
| MTOP, no examples | 76.0% |  |
| MTOP, 8 examples | 92.4% |  |

## Acting alone, at our 99% gate

- MTOP: Ren answered 21.0% alone and made 1 error in 923 answers. Its lower confidence bound is 99.40%.
- BANKING77: Ren answered 0% alone, because it couldn't prove it was safe. That gap is the research problem we're working on.

## Speed and cost

- GPU: 26 ms, not counting the network.
- Through the endpoint: 424 ms median from India to one GPU in the US.
- CPU: 2.18 s.
- Cost: $0.083 per 1,000 decisions at full load. Jev works out to about $0.008.

## Limits

- The result covers one seed with full history, and it doesn't certify a new business.
- When the right option was missing, Ren wrongly accepted 2 in 500, above its target of 1.
- No real business has run Ren yet.
- With no examples on BANKING77, Ren doesn't lead: Jev scores 79.7%.

## What's next

1. Build the GitHub connector.
2. Test Ren against Jev on real decisions.
3. Close the BANKING77 gap, so Ren can prove more of its answers are safe.