Making the mentoring process on Exercism a little more efficient

I just joined Exercism as a mentor a week ago, in the C# track, and after reviewing a dozen or so solutions I've started automating parts of the mentoring experience. Not any part that involves offering feedback! Just the manual steps to get to that point.

When you view a solution in the mentor dashboard, there's a command you can run that'll download it to a subdirectory of wherever you installed the Exercism CLI, along with a test file, readme, and other "stuff".

exercism download --uuid=cee00536fd614a4590918ce52a76bdd3

It prints the location to the console, but then it's up to you to open the right file and check things out. You don't have to download the solution, but as a mentor I think you should at least be verifying the tests pass - something that's not possible online right now.

Automate all the things!

So without further ado, here's a script I wrote that does a few things: (grab the gist on GitHub)

• Uses exercism to grab the solution with the given uuid, and captures the download path
• Gets around the odd \\ file naming issue documented here, here, and here by replacing the default .cs stub class with the one that starts with \\. Visual Studio can't even open the solution without throwing a "UNC paths should be of the form \\server\share" error unless you rename the file first.
• Enables all the unit tests by removing the (Skip = "Remove to run test") attribute. That attribute is there to encourage students to practice TDD and test one thing at a time. Unfortunately, that means mentors need to remove all those attributes each time in order to run the whole test suite.
• Sets DOTNET_CLI_TELEMETRY_OPTOUT to 'true' in order to opt-out of telemetry collection, and trust the ASP.NET Core HTTPS Development Certificate that gets installed - you'll see a notice about both the first time you run tests from the command line. (For setting an env variable in Windows, you'll have to make a change.)
• Runs the tests (which displays the results in the console) and opens up the project file. Make sure you check the code online first, in case some joker put something in there that'll do bad things when the tests run, like a File.Delete("some/important/common/file.ini")

Give it a simple name like "mentor", then add it to your path so you can run it from anywhere. Interestingly I didn't need to include dotnet restore to restore the XUnit NuGet package, but I'm unsure if that's due to VS4Mac or because running dotnet test does it for me.

mentor cee00536fd614a4590918ce52a76bdd3

If it works, you should see some output like this:

$ mentor a64e689427d447eaa5be1085f81bf691

Downloaded to
/your/path/to/exercism/users/johndoe/csharp/raindrops

Trusting the HTTPS development certificate was requested. If the certificate is not already trusted we will run the following command:
'sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain <<certificate>>'
This command might prompt you for your password to install the certificate on the system keychain.
A valid HTTPS certificate is already present.
Build started, please wait...
Build completed.

Test run for /your/path/to/exercism/users/johndoe/csharp/raindrops/bin/Debug/netcoreapp2.0/Raindrops.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.7.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

Total tests: 18. Passed: 18. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 1.4110 Seconds

I plan on keeping this up to date, as I find better ways to eliminate the manual steps. I want to spend more time helping people, not massaging each downloaded solution.

Next steps?

I've already noticed myself making some similar comments on the same exercise. I'd like to find a way to make notes about those frequent comments, and pull the right one down (from a git repo?) and have it right there in Visual Studio alongside the rest of the student's solution. (Hours after posting this, someone opened a similar issue on GitHub... maybe this'll be part of Exercism eventually.)

I might look for a way to add a button to the browser, which grabs the uuid from the address bar and launches the script directly... looks like I might need to read up on native messaging.

If you want to try out Exercism, join here ... and if you want to join me in being a mentor you can find out more about mentoring too.