Sign in to run and submit your work
Reading is open to everyone. Running code and saving drafts need an account so your work is yours and comes back on your next visit.
or
CODE WORKSPACE
After a failed run somebody has to decide whether re-running it is safe. That depends on what failed: timeouts yes, permission errors no, and a type the policy has never seen means you do not actually know.
Write classify_failures(failures, policy). Group the failures, expose the gaps in the taxonomy, and say whether to retry.
Function to write
classify_failures(failures: list[dict], policy: dict) -> dictA dict of failures grouped by class, the counts, the unknown types, and a retry verdict.
How to approach it
Add a failure type the policy does not list and read the advice.
Sample cases
+ 2 held back until you submit
a mixed batch with an unknown type
Three classes plus a type the policy has never seen, which must block the retry advice.
Input
Argument 1
| type |
|---|
| TimeoutError |
| TimeoutError |
| ValueError |
| NewThing |
Argument 2
{
'classes': {
'retryable': [
'TimeoutError',
'ConnectionError'
],
'fatal': [
'PermissionError'
],
'data': [
'ValueError'
]
}
}Returns
{
'by_class': {
'data': [
{
'type': 'ValueError',
'count': 1
}
],
'retryable': [
{
'type': 'TimeoutError',
'count': 2
}
],
'unclassified': [
{
'type': 'NewThing',
'count': 1
}
]
},
'counts': {
'unclassified': 1,
'retryable': 2,
'data': 1
},
'unclassified_types': [
'NewThing'
],
'retry_share': 0.5,
'should_retry': False
}everything retryable
The only case where retrying the whole batch is advised, and it has to be reachable.
Input
Argument 1
| type |
|---|
| TimeoutError |
Argument 2
{
'classes': {
'retryable': [
'TimeoutError',
'ConnectionError'
],
'fatal': [
'PermissionError'
],
'data': [
'ValueError'
]
}
}Returns
{
'by_class': {
'retryable': [
{
'type': 'TimeoutError',
'count': 1
}
]
},
'counts': {
'retryable': 1
},
'unclassified_types': [],
'retry_share': 1,
'should_retry': True
}a clean run
No failures is not a reason to retry, and the share must not divide by zero.
Input
Argument 1
[] (empty list)
Argument 2
{
'classes': {
'retryable': [
'TimeoutError',
'ConnectionError'
],
'fatal': [
'PermissionError'
],
'data': [
'ValueError'
]
}
}Returns
{
'by_class': {},
'counts': {},
'unclassified_types': [],
'retry_share': 0,
'should_retry': False
}Constraints
by_class, counts, unclassified_types, retry_share and should_retry.unclassified and is named in unclassified_types.unknown, which is then unclassified like any other.retry_share is retryable failures over all failures, rounded to 4 places, and 0.0 when there are none.should_retry is true only when there is at least one failure, no fatal ones, nothing unclassified, and every failure is retryable.Worked example
Four failures arrive: two TimeoutError, one ValueError and one NewThing the policy has never seen.
Half the failures are retryable, so a share-based rule would happily advise a retry. should_retry is false, because NewThing is unclassified — and an unknown failure might be a duplicate write, a partial commit or a corrupted file. The conservative answer is the correct one, and naming the unknown type is what turns the block into an action rather than a refusal.
What this tests
That a taxonomy has to make its own gaps visible, and that an automated decision built on it must fail closed. A classifier that silently buckets what it does not recognise is worse than no classifier, because it is trusted.
classify_failures(failures: list[dict], policy: dict) -> dictSubmit for review to find out what your query gets right, what it gets wrong, and how it compares with the best working query for this exercise.
This scenario runs a full workspace — editor, canvas and results side by side. It needs a laptop or desktop to be usable. Open this page on a bigger screen to start building.