Stack Overflow Talent had a saturated job board which resulted in a low number of views and a low number of apply clicks. We had a goal to increase our job distribution rate. It was my hypothesis that this could be done by creating a more effective way for users to search for and browse jobs. We also wanted to look at improving our job ads.
I approached this problem with a two part solition. The first part was a redesign of our job ads. A year earlier, another deigner attempted to do a full redesign of the job ads and saw a significant decrease in click thru rate. Being one of our most important ways to get users from Stack Overflow to the job board, we couldn't afford even a small dip. In order to combat this, I approached changing the ads incrementally.
I set up a calendar of possible changes we could make to the job ad. In this calendar, I laid out a an area of the job ad that we would test varients of. We would then take the winner and move onto the next change two weeks later. The areas of the job ad we were testing was Tag Varients, Footer, Varient style adjustment, Header, and Number of Listings
When looking back at the test that had failed before, one of the biggest changes was how the style of all of the ads became consistent. We had ads targeted at technologies such as Android that were styled in Android's iconic green. I had a hypothesis that this removal of technology branding in leui of Stack Overflow branding was hurting the ads because when you looked at the numbers, these ads had the biggest lose of click thru traffic. I made sure to focus specifically on these these varients which is why they have dedicated tracks in the a/b test calendar.