What Rose Petals Teach Us about Induction

by Oran Looney July 16, 2026 Philosophy   Machine Learning   LLM   Visualization

Richard Hamming famously used to ask his colleagues at Bell Labs this question: “What is the most important problem in your field, and why aren’t you working on it?” To which they probably replied, “Look, Dick, it’s eight-thirty in the morning. I haven’t even had my coffee yet. Who starts a conversation like that?”

Still, it’s a good question, isn’t it? For me, the answer is obvious: “Is there a general method for induction?” Now, you might think that the human race has a pretty good handle on induction, what with the scientific method and statistics and all that. But no: there’s a gap in the very foundation of our understanding, a gap we can only cross occasionally and haphazardly.

Hume called it the problem of induction; a catchier name is the No Free Lunch theorem, although it’s about as far from being a “theorem” as it’s possible to get. And it is simply this: there is no general, systematic way to go from observation to understanding.

If you haven’t encountered it before, it’s likely you don’t see what the big deal is. Don’t we all do this all the time, without even thinking about it? We do, but we don’t know how we do it, which means we don’t know how to teach it, how to automate it, or even if we’re doing it right.

What’s needed is a simple, concrete example which illustrates the idea without any particular need for mathematical sophistication. I’m going to give just such an example, show how various algorithmic approaches fare, and try to explain the unavoidable trade-off at the heart of the problem.

If all goes well, you’ll not only learn something important about one of the most fundamental unsolved problems out there, you’ll have developed a practical intuition that will help you understand, for example, why François Chollet introduced the ARC-AGI benchmark, why AI researchers keep talking about world models, and what it means to say the transformer architecture hits a sweet spot for language models.

The Game

Let’s start with a bit of nerd folklore, traditionally passed down by a friend who knows the trick and judges you to be the kind of person who will enjoy it. I’ve replicated the experience as nearly as possible here. You don’t have to solve it, necessarily, but you do have to make an honest effort if you want to fully grasp the underlying lesson.

All done? Great! What method did you use? Could you teach your method to a child? Could you write a program that implements your method? No? Well, I can’t either. That’s what this article is about. But let’s give it a shot anyway, using some off-the-shelf machine learning algorithms, and see if that tells us anything.

Spoilers Ahead!

Algorithmic Approaches

Here’s what I did. Different machine learning algorithms were shown a gradually increasing number of examples; each example consists of the numeric values of the five dice rolls as well as the correct number of petals around the rose for those dice. Each algorithm, in its own way, learns a function which maps those dice rolls to the number of petals. Then, its performance is evaluated on a different set of examples, ones it hasn’t seen before and wasn’t trained on. Whatever number the function outputs is rounded to the nearest integer and marked correct only if it exactly matches the true number of petals; if it’s off by even 0.5, it’s marked incorrect. The test accuracy is the proportion of dice rolls that it gets correct. Once a given algorithm has reached 100% accuracy, we deem that it’s saturated the benchmark and stop. The minimum number of examples needed to reach 100% accuracy is marked with a point and shown as the $N$ in the legend.

(By the way, the full source code and summary report are available on GitHub if you’re interested in the gory details.)

In a minute we’ll go case-by-case to try to understand why each approach fared well or badly, but first let’s just take a look at the high-level results:

Even at a glance, there are already some very interesting conclusions we can draw from the graph. First of all, it’s obvious that some of them solve it almost immediately, some struggle for a long time before noticing the pattern, and some never solve it at all. So there’s something that makes some approaches better or worse than others at this specific problem.

It’s also interesting that each curve seems to follow the same characteristic shape, even across fundamentally different learning algorithms. Each appears to require a certain minimum number of examples before it even starts to learn anything, but once it catches on progress becomes rapid and quickly reaches 100%. Perhaps this mirrors your own experience?

The obvious question, then, is why do different algorithms need different numbers of examples to “solve” petals around the rose? Is it just random? Is it opaque and unknowable, and we just have to try different approaches at random? Not at all.

Inductive Bias

The key concept is called inductive bias. A model’s inductive bias is, roughly speaking, a combination of its priors and its assumptions; that is, the kind of patterns it tends to prefer or be a good match for, and the structure that it requires any possible solution to have.

Models with an inductive bias that matches the specific structure of a dataset will learn it quickly, sometimes from just a handful of examples. Less specific, more general models with a larger hypothesis space to explore need far more examples to pick up on any patterns. And a model that has the wrong inductive bias or makes assumptions that directly contradict the true structure of the problem will never be able to learn anything at all.

Let’s dive into the specific details of a few of these algorithms to understand what exactly it is that makes them a good or poor fit for this problem.

Naïve Linear Regression

Let’s start with an intentionally weak linear model that serves as a baseline:

What makes this approach “naïve” is that it simply uses five numeric features, one per die.

Any ML enthusiast will tell you that’s stupid; but in a little bit we’ll see several other models that do just fine when fed the raw data in this format, so why should we spoon-feed the linear model?

You can see from the graph that this approach hasn’t made any progress, even after being shown 1,000 examples, but it’s actually worse than that: this algorithm cannot solve the problem. It never will; its inductive bias includes the assumption that the true model is linear in its parameters, which just isn’t true for this problem.

Consequently, this model is literally incapable of representing the true solution. I include it mainly as a warning for what happens when inductive bias goes wrong: make the wrong assumptions, and it’s not a matter of merely taking a little longer to catch on; you can end up with a model that just doesn’t work.

LR With Categorical Features

Of course, this is easily fixed using bog-standard feature engineering. One-hot encoding the dice rolls as categorical levels gives the model a better “vocabulary.”

Now it can treat each face as a separate case, and is able to find the solution. But not, as you’ll note, particularly quickly. The reason for that is simple: it’s treating each die separately, and has to “relearn” the rule for each die independently. Now, if the petals rule were something like “take the sum of the first, middle, and last die, ignoring the other two” then that would pay dividends: the model is flexible enough to learn different parameters for different dice. However, since that’s not how the rule works, this approach is slightly slower.

Bincount Pivot

There’s actually an even better feature representation that allows linear regression to solve the problem very quickly. Instead of treating each die as a feature, why don’t we have six features, each of which represents the number of dice showing a particular value?

def bincount(X: npt.NDArray[np.int_]) -> npt.

NDArray[np.int_]: return np.array([np.bincount(row, minlength=7)[1:7] for row in X])

This feature encoding makes the regression part trivial, and unsurprisingly it solves it with just $N=6$.

However, this doesn’t feel like the model is learning anything—we basically solved the problem for it. Once we have the idea to try this representation, we don’t even need to use regression, because the pattern is so obvious:

So, let’s look at some more general models that don’t feel like they’ve been hard-coded to solve this particular problem.

Fully Connected Neural Net

Neural networks are often sold as a way to avoid feature engineering. Sometimes that is true. Sometimes it just means asking a very flexible model to do the feature engineering for you, slowly, unreliably, and at great expense. That’s certainly the case here; the large, fully connected neural net we used is a very powerful, very general model, and it’s able to work from the raw features (without categorical encoding.) But it takes more than a thousand examples to solve it:

Note that this is after hyperparameter optimization: I let Optuna try hundreds of configurations—activation functions, layer sizes, learning rates—and out of millions of possible FCNN models, this was the best one. Let’s sit with that a minute, and really think about the decisions being made, and why they work.

First of all, sigmoid activation works best for this problem. That’s interesting because sigmoid is not often the best choice for modern deep learning networks, where “hinge” style activations like ReLU or SwiGLU usually work better. However, sigmoid makes it easy to construct step functions, so it wins here:

The other really interesting role of hyperparameters is tuning the overall “complexity” of the model.