After a week of testing and recording the data, there’s a clearer picture of what these bots are doing with these public commits or at least according to my tests in this particular setup. Listed below are each day’s set of total and unique clones, and what changing factor I made on each repository if any.
Strategy Alpha: Normal code with license
This test involved including a standard AGPL-v3 license file along with a small coded script with no comments. Each daily change involved superficial edits to the codebase sent as one commit per day.
| Day | Total Clones | Unique Clones |
|---|---|---|
| 1 | 9 | 9 |
| 2 | 16 | 10 |
| 3 | 12 | 7 |
| 4 | 8 | 6 |
| 5 | 9 | 6 |
| 6 | 19 | 10 |
| 7 | 10 | 6 |
Total Clones: 83
Unique Clones: 54
Strategy Beta: Nested markdown in directories
This test involved setting up a repository with an initial directory named ‘dir’ and subsequent directories named ‘.dir’ (note the preceding dot). Inside of a few of the ‘.dir’ subdirectories is a markdown file with a link to a uniquely renamed html file that is on my server. In this entire week of testing, no system visited any of the html links, so we know they are not following links. Each note refers to what the file was renamed to on both the markdown file and the server. This was also set as one commit per day, but the commit message was only limited to . as the message.
| Day | Total Clones | Unique Clones | Notes |
|---|---|---|---|
| 1 | 8 | 8 | _inner.html |
| 2 | 1 | 1 | _inner1.html |
| 3 | 17 | 11 | _inner2.html |
| 4 | 5 | 4 | _inner3.html |
| 5 | 11 | 7 | _inner4.html |
| 6 | 8 | 5 | _inner5.html |
| 7 | 14 | 8 | _inner6.html |
Total Clones: 64
Unique Clones: 44
Strategy Gamma: White-space changes and fifteen commits
This test involved making only white-space changes and always generating fifteen commits per day. Here is where we see what the bot systems are responding to more clearly—the number of broadcasted commits is more important than the content in the repository. That is, the source changes and file types aren’t relevant but the push to the public feed is. Each note refers to how the commits were organized on each test.
| Day | Total Clones | Unique Clones | Notes |
|---|---|---|---|
| 1 | 9 | 9 | all at once |
| 2 | 78 | 41 | separately |
| 3 | 70 | 23 | 3 x 5 commits |
| 4 | 13 | 7 | 5 x 3 commits |
| 5 | 4 | 4 | all at once |
| 6 | 65 | 22 | 3 x 5 commits |
| 7 | 43 | 12 | 5 x 3 commits |
Total Clones: 282
Unique Clones: 118
Strategy Delta: Newline changes and emoji commits
This test involved only making newline changes in a file with no extension and only using emoji for the commit message.
| Day | Total Clones | Unique Clones |
|---|---|---|
| 1 | 7 | 7 |
| 2 | 12 | 7 |
| 3 | 18 | 13 |
| 4 | 7 | 5 |
| 5 | 14 | 7 |
| 6 | 10 | 6 |
| 7 | 11 | 6 |
Total Clones: 79
Unique Clones: 51
Strategy Epsilon: Fourth wall commit messages
This final test involved making white-space and small text changes in a file with no extension, but the commit messages were always intended to break the fourth wall while ambiguously reading like a developer thinking out loud.
| Day | Total Clones | Unique Clones | Notes |
|---|---|---|---|
| 1 | 0 | 0 | Empty file |
| 2 | 20 | 15 | Added text in file |
| 3 | 8 | 4 | |
| 4 | 3 | 2 | |
| 5 | 24 | 15 | |
| 6 | 13 | 8 | |
| 7 | 12 | 7 |
Total Clones: 80
Unique Clones: 51
What to note about the Epsilon test is that once the file is more than 0 bytes, the bots respond to cloning at a higher frequency.
Possible Conclusions?
If I were to take an initial guess given the week of data above as a lens into what the greater picture is, then I would assume that the bots are listening to the public event feeds of developers on GitHub and when an event is pushed (such as a commit) and it also has a file with some bytes, it gets a higher likelihood of cloning.
We can see that the total and unique clones for each week of each test have similar values except Gamma, since it differs substantially in terms of the frequency of push events per day. The commit message content doesn’t seem to affect the cloning count much, except possibly that it seems to bias towards messages with more text versus not? Note how Beta’s test of only using . as the commit message while also having content changed in a markdown file.
Given the above assumptions, I suspect that these automated systems treat a repository as a higher signal if it has the following: constantly changing file sizes, changing file counts, minimally legible text in a commit message, and frequent push events—all of which lead to higher clone counts.