Hyperlinking logs to source code

Reading Time: 4 minutes

A lot of news came out of AtlasCamp 2016. I was especially proud of the story about TestFairy’s end-to-end integration. I helped TestFairy build the connections between JIRA, HipChat, Bitbucket, Bamboo, and Bitbucket Pipelines to their service for mobile beta testing. In their press release, TestFairy CTO, Gil Megidish explained:

Based on requests we’ve received from our large developer and testers community, we’re happy to release the solution that closes the loop between source code and crashes, by creating a link between the two. When showing a crash, TestFairy will automagically link the video recording of the user session and the stack trace to your Bitbucket repository, at the right branch and revision. Many of our customers who already tested the solution came back and said their work became much more efficient.

A stack-trace error in TestFairy hyperlinks to the specific line is Bitbucket source code

I hope to see other integrations follow TestFairy’s lead and hyperlink to source code in Bitbucket. Fortunately, it is relatively simple to implement with some forethought. The hyperlink itself requires 7 parameters.

Key Description Example Source
bitbucket The base URL for Bitbucket Cloud. https://bitbucket.org static,
or build server
owner The account owner for the repository that contains the target file. atlassian build server,
or associated to log
repository_name The name of the repository that contains the target file. Also known as the repo slug. python-bitbucket build server,
or associated to log
commitish The ref (branch or tag) or commit SHA for the state of the target file. master build server,
or associated to log
filepath The path within the repository and the name of the target file. tests/test_auth.py log message
fileviewer The file viewer for the target file. If not provided, Bitbucket will append the default file viewer to the URL. file-view-default static,
or associated to log
fileline The target file name plus the line number, separated by a dash. Without the fileline segment, the URL simply navigates to the target file. test_auth.py-29 log message

When you assemble these parameters, you get a URL like this (some spacing added for readability):


https://bitbucket.org/atlassian/python-bitbucket/src/master/tests/test_auth.py
    ?fileviewer=file-view-default
    #test_auth.py-29

The trick is that different parameters can come from different places.

Static

The easiest source is what I labeled static. In other words, it’s a value that could be hard-coded into an integration, without causing long-term maintenance pain. This URL pattern only works for Bitbucket Cloud; hence, the base URL can be harded-coded to the production URL for Bitbucket Cloud. The file viewer is an optional parameter so it could be left out entirely. The safest file viewer would be the default, since any other choice would require every user who clicks the link already has that file viewer.

Build server

The build server should know the “provenance” of a build and deployment. It should be able to forward the source context to the log server. Let’s look at some more concrete ways that might happen.

For TestFairy, we used specific parameters on the deployment hook to remember the source context from the build environment. Consider the following lines of Bash:


# The following accounts for paths in the repo 
# that are stripped as the Java source code is compiled into a JAR.
BITBUCKET_SRC_PREFIX='app/src/main/java'

# The following accounts for paths created by the build script.
APK_FILE="./app/build/outputs/apk/app-debug.apk"

# Slashes create one variable 
# with a comma-separated list of key=value pairs.
# The variables are automatically provided by the build server,
# in this case Bitbucket Pipelines.
CUSTOM=ci=bitbucket,
repoOwner=${BITBUCKET_REPO_OWNER},
repoName=${BITBUCKET_REPO_SLUG},
filePrefix=${BITBUCKET_SRC_PREFIX},
commitHas=${BITBUCKET_COMMIT}

# The TestFairy API key is managed by Bitbucket Pipelines
# so that it can be kept a secret.
curl https://app.testfairy.com/api/upload 
    -F api_key=${TESTFAIRY_API_KEY} 
    -F file=@{APK_FILE} 
    -F custom=${CUSTOM} 
    -F format=readable

Although this example used specific variable names from Bitbucket Pipelines, the same information is available in other build servers like Bamboo and Jenkins. The same information can be forwarded in different ways. For Java JARs, it could be written into the META-INF/MANIFEST.MF. Or maybe it could be provided via a naming standard for the JAR itself, like: repo_owner-repo_slug-commit.jar

Associated to log

If not provided by a build server, an approximation could be provided by a user for a given log. A user can easily find the owner and repository name. The commit SHA might change frequently, so instead the user might provide the branch where source is most closely related. With common Git workflows, that is likely to be master.

While it might be easier to obtain these details from a user, the downside is there may be subsequent commits on the target branch, making the pointer into the file either invalid or wrong.

Log message

The details of filepath and fileline will usually be found in a stack trace. The trick here is to parse the file path from the stack trace and to know that file’s relative path in the source repository. For Java projects, Maven directory conventions will inject a few extra directories. That’s why the TestFairy approach sends that information explicitly. For other languages, the mapping to source code is often easier because mapping to the source path follows a simpler convention.

What are you waiting for?

I hope you’ll find ways to weave in hyperlinks back to Bitbucket. If you have questions or need other help working with the Bitbucket endpoints, tweet me at @devpartisan or my team at @atlassiandev.