Secondary Research
The project is based on 4 hypotheses. Therefore, in my secondary research, I looked for existing data, studies and articles from authority sources to validate each of the hypotheses to support the project basis.
(The 4 project bases)
1. There’s an increasing number of film students looking to enter the workplace.
According to Data USA, there’s been an overall growth in the number of students receiving film degrees in the US over the past few years. In 2020, the number has grown by 5.03%, with a total of 8,429 students graduating from film institutions in the US.
%20(1).png)
(Number of students receiving film degrees in the US from year 2012 to 2020. Data from Data USA)
Many other countries have also witnessed a significant increase in the number of individuals in the workplace with a Cinematography & Film degree. As there are more and more people graduating with a film degree, the number of job seekers in this field would naturally increase. These individuals need resources and assistance to enter the workforce.
2. Young filmmakers are facing extraordinary financial challenges
Colleges are expensive, but even more so with film schools. While tuition at a film school can be significantly higher than traditional colleges and universities, it also charges extra fees that are not included in the tuition.
(An example of a half-year tuition bill from a prestigious film institution in the US, 2018-2019. Anonymous contributor)
Due to extraordinary financial burdens, a dominant group of film students take out student loans to help cover their study costs. For example, at Los Angeles Film School, “56.0% of incoming students take out a loan to help defray freshman year costs”, and “the average loan amount for all undergraduates is $8,425 per year.” (Los Angeles Film School Loan Debt: How Much Debt Do Students Graduate With?)
3. There’s a gap between the world of film school and that of the industry
The existing crewing platforms do not give special consideration to the needs of student filmmakers. From unpaid roles to 3k-weekly-salary positions, from entry-level to senior-level jobs, those platforms encompass an audience so broad that student filmmakers are forced to compete with individuals who are way more experienced and advantaged. They blur the line between those who make money through making films and those who make money to make films.
(Examples of the casting calls on Backstage)
4. Democratization of filmmaking tools is the trend
The past decade has witnessed the legendary Youtube revolutionize the world of new media, making the tools for filmmaking and distribution much more accessible to ordinary people. Most recently, Tiktok has started another wave of democratization of filmmaking. With over a billion active users, Tikok’s low barrier for entry allows inexperienced yet passionate individuals to be creative and hands-on.
(Growth of active monthly users on Tiktok since its launch in 2018. Data from Backlinco)
Likewise, a platform that makes film production more accessible will also be welcomed by creatives with limited budget and experience.
Conclusion
With reliable data and reports, I've validated the project basis that young filmmakers are in need of a crewing platform that especially tailors to their needs and wants. I believe that a product like Crewmate will be in great command and helpful for many aspiring filmmakers who are struggling to enter the industry.
See full secondary research report>
Competitive Research
At the earlier stage of my competitive research, I compiled a list of strong competitors that specialize in crewing / hiring services and compared their features. The only two features that are shared across all platforms are the Post Jobs and Find Jobs, while other features occur occasionally on a select few of platforms.
(Competitor feature list)
Later in the survey phase, I asked people about the practicality of all these features, and surprisingly learned that only a very few of them are actually useful to the users.
Out of all the competitors on the feature list, I chose the top three that have the most users to conduct an in-depth heuristic analysis of their design, hoping to learn from their strengths and weaknesses. The chosen companies are Backstage, Mandy, and Production Hub. I used 4 out of the 10 usability heuristics from the Nielson Norman Group’s 10 Usability Heuristics for User Interface Design.
(Usability heuristics rating results)
The 3 important takeaways from the heuristic evaluation are:
1. Consistency is the key. While all three products in general perform comparatively well in some aspects, only Backstage shows consistency in its usability across all four heuristics being evaluated.
2. Aesthetic and Minimalist Design is the weakest spot. Both Mandy and Production Hub performed extremely poorly in the Aesthetic and Minimalist Design heuristic. Production Hub could have otherwise measured up to Backstage.
3. Usability before aesthetics. Despite its beautiful UI design, Mandy has major usability problems in all four heuristics that hinder the overall user experience.
See full heuristic evaluation report>
Survey & Interview
“Aspiring artists urgently need a platform to find people like their own to collaborate with.” — survey participant
In order to gain more quantitative insights about the potential users to understand their demographics and psychographics, I conducted a screener survey online through social media and film forums. The survey starts with three screening questions, followed by three main sections concerning one’s background, work experience and crewing platforms usage respectively.
The survey yielded 51 total responses and 31 qualified responses. The results have validated the assumptions that young filmmakers are in tremendous need for professional connections and job opportunities, and that limited budget plays an important role in the hiring process.
(Online survey report)
To gain more qualitative insights about the potential users and identify their needs, wants and motivations, I selected 8 participants from the survey to conduct a 40-min interview with each of them. 4 of them are employees and 4 of them employers.
(Affinity map: unsorted)
Their inputs are then organized into 5 categories: 1) Criteria for crew employer, 2) Criteria for employee, 3) Criteria for project, 4) Attitudes towards pay, and 5) Attitudes towards crewing platforms.
(Affinity map: sorted)
These inputs have proved wrong some previous assumptions. One of them is that unpaid students are usually less skilled than working professionals. Yet, many interviewees reported that students often times do a better job than paid professionals, and that’s because they devote more passion, time and effort to the project.
(Empathy map: the employer)
Another false assumption is that crew members would work for free only if they are fond of the project. However, in fact, a lot of crew members don’t have options to choose from and often end up working on projects they don’t feel passionate about.
(Empathy map: the employee)
“It’s all about the vibe of the team.” — anonymous interviewee
There are some unexpected pointers that also came up during the interviews:
• Word of mouth plays an extremely important role in the crewing process
• It’s not just about the individual but also how they fit in the team
• Knowing someones’s personality is as important if not more so than knowing their work
All these findings have greatly informed my design decisions later in the prototyping stage.
Synthesis
“It’s pretty much like marriage, really, to find someone you’d like to work with.” — anonymous interviewee
After interviewing various people, I realized that I had made an important mistake: The ecosystem that I originally drew was inaccurate.
The fact that the director wants to gain help and support while the crew member wants to gain experience and growth makes them collaborators instead of employers versus employees. Instead of the traditional top-down recruitment process, the crewing process among student filmmakers is more like finding the people that you vibe and want to make cool stuff with, regardless of their title.
(A shift in the perspective of the ecosystem)
Enlightened by this finding, I identified the core values of this app: equality, mutuality and collaboration. This not only informs the design, but also the branding, marketing and every other aspect of the product. Moreover, it helps to make our app really speak to its potential users and stand out in the market, where most competitors frame their users as being in a hierarchal employment relationship.
I want to combine the features of hiring platforms and social media in this app, making it an incubator for professional connections and collaborations. I hope to empathize on the social and networking nature of it, removing the hierarchy that normally comes with traditional hiring processes.
(Primary persona: the project starter)
To reflect the change in the ecosystem, I changed the personas from “employer” and “employee” to “project starter” and “crew member”, respectively. I also decided that "project starter" be the primary persona while "crew member" the secondary, since the product initially set out to help underfunded filmmakers to find crews.
(Primary persona: the crew member)
What the two personas share in common is their concern about financial issues. Alice (project starter) is worried that her limited budget would make it hard for her to find the ideal crew members or to finish making her film. Nick (crew member) hopes that there’s enough transparency about compensation and other money matters in the crew calls.
The main difference between these two personas is their prioritized values. Alice prioritizes safety, smoothness and completion of the project, while Nick prioritizes experience, growth and networking.
After synthesizing all the primary research findings, I refined and polished the problem statements in the format of HMW questions:
1. How might we provide a platform for aspiring filmmakers to connect with each other?
2. How might we make the teaming process more equal and transparent?
3. How might we build more trust between filmmakers who don’t know each other?
With these questions in mind, I moved on to the ideation and prototyping stage.
Ideation
To kick off the ideation phase, I first created a list of user stories in an excel form based on the findings from primary research. These user stories are arranged in the hierarchy of epics, stories and tasks. Then, I picked the top three epics to be included in the MVP and mapped them out in a user story map. These three epics are:
1. As a student director, I want to look for a passionate and skilled crew member, so that I can use their help in my project.
2. As a student crew member, I want to find an interesting project to work on, so that I can gain more set experience and enrich my portfolio.
3. As a student director, I want to direct message a crew member, so that I can ask about their availability.
In the next step, I created a sitemap to keep track of all the elements within the app and visualize their connections.
(Sitemap)
In the next step, I identified the red routes that I’ll be focusing on, and sketched out their user flows.
(User flows: red routes)
Prototyping & User Testing
Prototype 1: Sketches
The first prototypes I made are paper prototypes, which allowed me to sketch out ideas quickly and make changes at low costs. Moreover, doing user testing with paper prototypes forces the users to focus on the functionality rather than appearance of the design, which is exactly what I wanted the most from their feedback at this stage.
(Paper prototypes)
Guerrilla Testing
To learn about people’s first impressions of the interface and discover usability issues, I conducted a guerrilla test with 5 participant using the paper prototypes. Each of them was given 3 tasks to complete. They were asked to think aloud throughout the entire test, while I took notes on the side, only occasionally jumping in to ask them to articulate their thoughts and decisions.
There are three common issues that came up during the test:
• 4 out o 5 people can't tell the difference between the "Connect" and "Message" buttons.
• All of them don't know what the "Choose a Project" button means when reaching out to a crew member in task 1.
• All of them can't figure out the “correct” way to apply for an open role in task 2. That is, no one went down the path that I designed for them to complete this task, and no one chose to click the drop down tab to reveal the hidden job description for each role.
What went well during the test are:
• Everyone found the message feature rather intuitive and easy to use. 4 out of 5 participants took advantage of the advanced search function and used quick filters to accelerate their search process.
• People enjoyed the simplicity of the interface and the comprehensiveness of the functionality.
Read full guerrilla testing report>
Prototype 2: Wireframes & High-Fidelity Screens
I made some revisions based on the feedback from the guerrilla user test and moved to the digital space to create wireframes.
(Wireframes)
After having the skeleton in place, I created a mood board and style guide that sets a lively, young and professional tone, reflecting the demographics of the users.
Then, I created the high-fidelity clickable prototypes in Figma.
(High-fidelity prototypes)
Moderated User Testing
After completing the high-fidelity prototypes, I conducted another around of user tests to uncover more usability issue. 5 moderated user tests were conducted with 5 selected participants, each of whom were asked to complete the same tasks as before using the clickable prototypes.
There are some new issues that surfaced from this user testing:
• Confusion around navigation. Users are unsure where in the app they are located; the difference between “Search” and the “Sort” functions is unclear;
• Ambiguous terms. Users aren’t sure about the meanings of the crew tags under each project thumbnail; the metadata of the film in the project page are hard to comprehend; the term "Pending" is confusing.
• Some important information is missing. Users want to see a crew member's reel inside the app instead of through an external link; some want to see how many credits a crew member has more than how many mutual contacts they have.
People’s most positive feedbacks are around the aesthetics of the product and the design of certain small features including the advanced search functions, group chat feature and the mutual connection feature.
Read full user testing report>
Prototype 3: Revised High-Fidelity Screens
Based on the second round of feedback, I redesigned the screens to improve the information architecture and consistency, especially in terms of hierarchy, grouping, and the Recognition rather than Recall principle.
I focused on improving on the following aspects:
1. Design a better tab bar that indicates clearly where the user is located.
2. Use better grouping techniques to separate the “Search” and “Sort” functions.
3. Create a better information and visual hierarchy in the individual profile/project pages.
In the end, I conducted a 3rd round of user tests with 5 new participants, assigned with the same tasks as before. Their task performance has greatly improved as they show more clarity in their understanding of what’s going on and confidence in what they are doing.
Reflection
Looking back on the entire process, there are a few things that can be improved upon. During the screener survey, the screening questions could’ve been crafted in a smarter way so that they don’t screen out some potentially qualified participants. For instance, someone who has a full-time job could also be a potential user of this app and thus need not to be screened out, since many independent filmmakers make films outside of their daily job. Such issue can be avoided in the future with a more rounded secondary research.
Another thing I wish I have done is to conduct a round of user tests with the wireframes before making the high-fidelity prototypes. It’s most economic to discover as many usability issues as possible earlier in the design process, and therefore it's best to take the time to refine the design with lower-cost prototypes. Although a user testing was done with the paper prototypes, there were still some major usability issues that weren’t discovered then, and it’s more difficult and time-consuming to fix them in the high-fidelity prototypes than in the wireframes.
What I found extremely fascinating is that the primary user research helped me build a more accurate ecosystem that our competitors didn’t quite capture — this could be an important opportunity for our product to find its niche in the market and speak to its potential users. I could not have come to this finding without conducting extensive interviews with the right participants who represent different voices. This again shows me the importance of a meaningful, quality user research and that of asking the right questions.
