Survey on Interaction Between ML Developers and Domain Experts
This survey is intended for developers and engineers who create machine learning models to support decision-making for domain experts. For example (but not limited to):

- Medical decisions for diagnoses, trials, procedures.
- Business and financial decisions about loans, investments, acquisitions.
- Failure predictions in manufacturing and computing infrastructures.
- Candidate screening / hiring in recruitment or admissions.
- Etc.

We are a group of researchers from NYU and Northwestern who are interested in understanding how model developers and engineers interact with domain experts when developing ML models in these contexts, and their goals in doing so.

By completing this survey, you will:

- help us create solutions to better support ML model developers
- be eligible to win one of multiple $50 Amazon gift cards
Sign in to Google to save your progress. Learn more
Please describe your job role and the industry you work in.
To what extent do you work with domain experts in developing a model? Please describe why and how often you consult with domain experts in the model lifespan (including after deployment if necessary).
Who are the domain experts and what decision are they making?
What are your goals in consulting domain experts? Check all that apply:
If you have any additional details that you'd like to add on your goals in consulting domain experts, please add them here.
Please explain the process you use to communicate with the experts and gather their feedback. Be specific.
What forms does your communication with domain experts take? Check all that apply:
In model development (prior to deployment), how often would you say you typically interact with domain experts to get feedback?
Clear selection
What tools and representations (graphs, tables, written summaries, etc.) do you use to summarize a model that you want to get expert feedback on? What information is included in these representations? E.g. if you use graphs, what are they of?
How do you record the feedback you get from domain experts? (e.g., note-taking, tables, ….). What information does this feedback contain? Please describe in detail.
How do you use the information you get from domain experts in the model development process?
Can you think of anything that would make your interactions with domain experts easier? Could be a tool, a more systematic method, etc.
Imagine a tool that enables you to collect domain experts' feedback on a machine learning model and its predictions. The tool elicits information from the experts and guides them towards exploring regions where the ML model may conflict with their intuition. If such a tool existed, a) what would you want to elicit from the experts? and b) how you would like to use it?
Would you be willing to be interviewed by our research team for approximately 30 minutes about your goals and process for communicating with domain experts? If so, please add your email below. Your feedback will guide our development of tools to help improve the processes by which ML developers and domain experts interact.
Add your email below if you would like to participate in the lottery for a $50 Amazon gift card (1 raffled for every 20 participants)
Submit
Clear form
Never submit passwords through Google Forms.
This form was created inside of New York University. Report Abuse
Google Forms