Chip Li李鸿铭

Gainesville, FL

What do people need before AI acts for them? · Pro-CaRE · 2023–2025

Recommend what students grew from, not what they clicked

Which internship should an engineering student choose, and will they trust an AI’s answer?

My role
Third author and lead developer of the system, working with the research team.
Method
A survey of 223 mechanical and aerospace engineering students about their internships, tasks, learning, and self-efficacy; 1,913 tasks extracted from job postings with an LLM and classified by fine-tuned models; a hybrid recommender tested on held-out students.
Key insight
Internships differ most in what students do day to day. Matching on tasks and reported growth predicts outcomes well, and new postings can be recommended the day they appear.
Outcome
R² above .72 across six outcomes on held-out data; interviews on the explanations are under way.

Product ↗Paper ↗

Pro-CaRE home: Engineer Your Career Path

Problem

Students won’t hand a career decision to a black box

Internships and co-ops shape engineering careers, but advising is uneven and postings churn every week. A recommender could help, but only if it optimizes for the right thing and students can question it.

Approach

Ask students what they grew from, then describe every job by its tasks

  • Measure growth, not clicks223 students reported their tasks, technical and professional learning, and four facets of engineering self-efficacy: confidence, leadership, identity, and commitment.
  • Turn postings into tasksAn LLM broke internship postings into 1,913 concrete tasks. People labeled them, and fine-tuned BERT-family models sorted them into 15 categories.
  • Recommend on predicted growthA hybrid recommender predicts each student’s growth for each opportunity. New postings work immediately, before anyone has rated them.
  • Explain every recommendationThree kinds of explanation: why this one, which features mattered most, and examples of similar students.
Pro-CaRE recommendation view: student profile, recommended internship with growth potential, and predicted growth across six outcomes with explanations
A recommendation explains itself: predicted growth on each outcome, and what each score means.

Findings

What students actually do in internships

223students surveyed
1,913tasks extracted from postings
R² > .72across six outcomes, held-out students
Tasks students reported doing in their internships (n = 223)
On-the-job technical training181
Working with a team166
Design and development150
Written documentation149
Analyzing system performance143
Oral presentations143
Quality control, troubleshooting96
Writing code or tools87

Next

The question that decides adoption: do students trust it?

The explanation study is under way: interviews with students about whether each kind of explanation helps them judge a recommendation and decide whether to act on it.

if I ran it again

The sample covers two majors at one university, and learning outcomes are self-reported. So far the evaluation is predictive accuracy. The user-facing question, whether explanations change trust and choices, is the part I most want to answer.