Cheating the contest – Brian Welch

Posted by Helen Chen,

Partway during the contest, an interesting series of entries by Brian Welch appeared in the queue. These entries were a clever attempt by Brian to obtain a higher score by tricking the contest machinery into giving it more guesses. Here is Brian’s discussion+ of the contest entry:

Use of MLOCK to obtain extra guesses

The inspiration for attempting this “cheat” came from the test suite itself which uses a persistent variable to keep track of the number of calls made to the SCOREME function. SCOREME contains the following code:

persistent numCallsMade
if isempty(numCallsMade)

The persistent variable numCallsMade is accessible only to this function, and its value is retained in memory between calls to the function. Persistent variables are known only to the function in which they are declared and cannot be changed by other functions or from the MATLAB command line (unlike global variables, which all exist in a common global workspace). The only way to clear a persistent variable is by using the CLEAR command. For example, the RUNCONTEST function in the test suite uses the following command to reset the SCORME function before testing each individual puzzle:


MATLAB also contains a command called MLOCK, which is designed to prevent a function that has been loaded into memory from being cleared (you can use INMEM to see what functions are currently loaded into memory). A consequence of MLOCKing functions is that their corresponding persistent variables are also protected and will not be cleared.

Because a critical feature of this contest was the limited number of guesses available for each puzzle, a submission that could somehow resume calculations where it ended in a previous run could significantly improve its score. To guarantee that the submission M-file would not be cleared and lose its persistent variables, the MLOCK command can be used. This scheme could only be thwarted if the contest queue machinery either did not allow the MLOCK command to appear in submissions (as was eventually done) or if the MUNLOCK command was used to force the submissions to be clearable.(Contest Team Note: The MLOCK command was initially left off the list of disallowed functions; it was added once Brian’s code was spotted)

In the Persistence0x entries, with Salvador Dali listed punfully as the author, the following lines appear:

persistent puzzles bestScores

puzzles is a structure that preserves the highest scoring guess while bestScores contains the scores to beat. Both of these variables can be indexed by the puzzleId input argument. Because these variables aren’t cleared between runs, they can be used to store guesses, giving each entry a larger set of guesses from which to choose the best entry. This allows submissions with the same exact code to get higher and higher scores. For example, the Dali series (still appearing in the complete rankings as passing entries) is the same file run 5 times with the black peg percentage ever increasing: 34.19, 44.69, 50.78, 54.00, 56.44. A more sophisticated entry could have made better use of the additional guesses and likely approached near perfect scores in a small number of submission iterations.

The drawback of using the MLOCK function is that it was impossible to discontinue its activity by issuing an MLOCK command in a subsequent entry because the new submissions would never replace the locked version. This eventually required the queue to be reset with MLOCK added to the prohibited function list.

Brian Welch

Brian’s entries certainly were clever. As he pointed out, to correct this we disallowed use of the MLOCK function and rescored entries received after his.

* Some edits were made to the original author’s description


Comments are closed.