Psychometrician, AWS Training & Certification - Arlington | Amazon Job

Job Description

As part of the AWS Applied AI Solutions organization, we have a vision to provide business applications, leveraging Amazon's unique experience and expertise, that are used by millions of companies worldwide to manage day-to-day operations. We will accomplish this by accelerating our customers' businesses through delivery of intuitive and differentiated technology solutions that solve enduring business challenges. We blend vision with curiosity and Amazon's real-world experience to build opinionated, turnkey solutions. Where customers prefer to buy over build, we become their trusted partner with solutions that are no-brainers to buy and easy to use.

The AWS Certification team is seeking a Psychometrician with experience working with criterion-referenced assessment programs to support a large global AWS Certification and Credentialing program. In this role, you will support all psychometric aspects of exam development and operation, including job analyses, standard setting, automated test assembly, item and test analyses, optimal item bank design, quality assurance, and project planning.

You will work closely with a team of psychometricians, subject matter experts, certification exam program managers, publishing, delivery, security, and product management teams to support ongoing analyses of exam and credential data. To be successful in this position, you must be highly motivated, creative, detail oriented, and a self-starter who is able to think big, execute, ensure high quality, yet stay focused on the details.

Key job responsibilities
-Conduct Job Task Analysis (JTA) workshops and post-JTA survey analyses to define the blueprint and test specifications for new certifications or updates to existing certifications

-Conduct standard setting studies to set the passing score for exams and credentials

-Run item analysis to evaluate quality and performance of exam items

-Use automated test assembly procedures to assemble forms or item pools

-Work with content development to track item bank trends and optimize the health of item banks

-Support the development of a cloud-based analytics and reporting system

-Partake in development and performance analysis of credentials

-Interpret and clearly communicate the results of analyses to stakeholders through written and oral reports

-Follow the accreditation standards set by ISO/IEC 17024:2012 and the National Council for Certifying Agencies (NCCA) as they relate to valid psychometric practices

-Contribute to the development and execution of the strategic goals regarding the AWS certification and credentialing program

-Consult with leadership, internal staff, external consultants, and industry leaders regarding advancement of current offerings

A day in the life
In this role, you'll partner daily with fellow psychometricians, subject-matter experts, certification exam program managers, and the publishing, delivery, security, and product management teams to support the AWS Certification and Credentialing program. A typical day includes running item and test analyses, facilitating Job Task Analysis workshops, conducting standard-setting studies, and using automated test assembly to build forms and item pools. You'll monitor item bank health, interpret results, and communicate findings to stakeholders through written and oral reports—all while ensuring exams meet ISO/IEC 17024 and NCCA accreditation standards and advancing the program's strategic goals.

About the team
Diverse Experiences Amazon values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn't followed a traditional path, or includes alternative experiences, don't let it stop you from applying.

Why AWS Amazon Web Services (AWS) is the world's most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that's why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.

Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there's nothing we can't achieve in the cloud.

Inclusive Team Culture Here at AWS, it's in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity and AmazeCon conferences, inspire us to never stop embracing our uniqueness.

Mentorship and Career Growth We're continuously raising our performance bar as we strive to become Earth's Best Employer. That's why you'll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.

Basic Qualifications:

- PhD
- At least 1 year post-doctoral direct experience working with high stakes certification or licensure programs
- Experience conducting job task analysis and standard setting
- Experience in equating, item analysis, and form assembly
- Experience using item response theory
- Experience using Generative AI tools
- Proficiency in R or Python languages

Preferred Qualifications:

- Experience in testing for global audiences
- Experience with different exam designs and delivery (e.g. LOFT, CAT)
- Experience using advanced AI methods (e.g., Machine Learning, Agentic AI)
- History of conference presentations and publications
- Experience with data forensics methods

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.

USA, GA, Atlanta - 136,000.00 - 184,000.00 USD annually
USA, TX, Austin - 136,000.00 - 184,000.00 USD annually
USA, VA, Arlington - 136,000.00 - 184,000.00 USD annually
USA, VA, Herndon - 136,000.00 - 184,000.00 USD annually
USA, WA, Seattle - 136,000.00 - 184,000.00 USD annually