An unsure prompt-injection flag no longer stops a task
Sinatra's prompt-injection screen now stops a run only when its classifier is at least 80% sure, so an ordinary reply isn't stopped on a coin flip.
An ordinary reply, asking Sinatra to stop a test and report its last line, scored 54% likely to be an attack, and the run was stopped. Because every later reply on the issue re-checked that comment, the task then stayed stuck. Below 80%, a flag is now logged and the run goes ahead. That's a small loosening: an attack the classifier is only moderately sure about would get through, though every attack in our test set scored above 80%. A comment it is sure about still stops the run, and keeps stopping later replies until it's removed. The security page now lists this next to the existing case where a classifier error lets a run go ahead.
Update: within a run, a flagged reply no longer stops the run. It is refused on its own and the task keeps waiting. A GitHub, Slack or API reply that arrives after the run went idle still stops the task. On Linear the comment stays on the issue, so while it is among the issue's recent comments later requests are refused the same way, and a fresh run that finds it is still stopped, until it's edited or removed.
#1619