25-45 Examinations Manager

APEGA • Edmonton • 1w ago

Want to be part of a team that makes a difference? Come be part of the change. Since 1920, APEGA has been regulating the practices of engineering and geoscience to serve the public interest in Alberta. We are looking for energetic people who thrive in a dynamic, fast-paced, and challenging environment. You are committed to serving the community with integrity, accountability, and innovation, and capable of delivering exceptional service. If you reflect these values, have the knowledge, skills, and abilities to make a difference through your work, and are passionate about acting in the best interest of public safety – then come be a part of our diverse and inclusive team! Job Title:

Examinations Manager

Job Family:

Manager

Department:

Registration Examinations

Reports To:

Director, Registration

Division:

Regulatory

Location:

Edmonton

Competition:

25-45

Employment Indicator:

Permanent, Full-time

Position Summary The Examinations Manager is responsible for the delivery of provincial and national, high-stakes, academic and licensure examinations on behalf of APEGA and other provincial engineering and geoscience associations in Canada. These include the National Professional Practice Examinations (NPPE), technical examinations, National Association of State Boards of Geology (ASBOG) exams (Fundamentals of Geology (FG)), and National Council of Examiners for Engineering and Surveying (NCEES) exams (Fundamentals of Engineering (FE), Principals and Practices of Engineering (PE), Structural Engineering (SE)). The Examinations Manager leads the examinations program through robust planning and execution, contributing to policy development, goal setting, and financial and human resource management. The role is accountable for the training and oversight of the examinations team, several dozen exam question writers, committee members (both volunteers and paid contract staff) and vendors who assist in delivering the examination programs. The Examinations Manager will utilize the current examination data storage system to support registrant database access, item authoring, item banking, and exam delivery. They will work closely with Examination Administrators and vendors to support exam administration tasks such as organization of materials, and registration management. This role is integral to the successful development and delivery of APEGA examination programs and to the relationships with other Canadian regulators, partners/vendors, applicants, and licensed professionals. This role requires a high level of diligence and attention to detail to ensure accuracy of reporting, examination decisions and continuous improvement.

Responsibilities Team management, financial management, contribute to policy development, and program development and delivery including administration of the examination process, issues and change management. Manage the Examinations team through training and onboarding of newly hired staff on policies and procedures based on competencies of role. Guide and coordinate staff, delegate tasks ensuring effective day-to-day operation. Develop and maintain procedures, tools, improvements to ensure consistent execution of goals set by the Director, Registration and for the team. Provide feedback and development opportunities for team, and complete performance evaluations and year-end reviews. Delegation of a subset of non-technical tasks related to the administration of the exam process to the Exams Administrators who coordinate and process applicant exam registrations, testing accommodation requests, exam test center logistics, and maintain applicant records in the database. Ensure that resources aimed at exam candidates, including tools and information posted for their benefit on the APEGA and NPPE standalone websites, are current, appropriate, and effective. Ensure that applicants and exam candidates are eligible for examinations registered for and have paid the fees prescribed by the Council, follow up on outstanding invoices, . Non-technical tasks in this context are delegated to the Exams Administrators, with the Examinations Manager handling the escalated cases. Work with third party exam proctoring vendor to ensure candidates’ requests are granted. For example, provide recommendations to the vendor, coordinate with partner regulators, and decisions on the set-up of non-standard examinations for special accommodation candidates. Respond to inquiries from applicants when they are not satisfied with the information provided by the Exams Administrators. Work with an external contractor to conduct all technical scoring, reporting, and psychometric analyses on the NPPE and analyses, as required, for all other exam programs. Ensure that exam results are promptly communicated to candidates, notify the BOE and partnering associations, and complete registrations of successful candidates. Non-technical tasks in this context are delegated to the Exams Administrators with the Examinations Manager handling the escalated cases. Work with an external psychometric contractor to recommend policy and or psychometric approaches or procedural changes (content development and review, security strategy and monitoring, cut score setting, etc.) to BOE and partner associations to ensure continuous improvement and alignment of Examinations with the needs of the associations and the profession. Utilize psychometrics, the assessment industry standard scientific approach to create valid, reliable, and fair exams that test the required knowledge, skills, and abilities (competencies) to be licensed as a professional engineer/geoscientist. The role works with an external contractor to maintain and enhance the NPPE program and a bank of questions, and work with item writers to develop test questions on a variety of competency areas (ethics, law, professional practice, regulation, etc.). The role collaborates with an external contractor to support psychometric functions related to the NPPE, including: Ensure the exam program meets standards for validity, reliability, and fairness as outlined in the Standards for Educational and Psychological Testing Oversee the selection, use, and staff training for item authoring, banking, and exam administration systems Support item writing, revision, and review to maintain a robust item bank for future exam forms Create and validate NPPE exam forms and coordinate blueprint updates with NPPE Advisory Committee approval Maintain Service Level Agreements (SLAs) with participating regulators Retrieve, score, and analyze candidate responses, and generate individual mastery reports for unsuccessful candidates Conduct test security and psychometric analyses (e.g., collusion detection, DIF, standard setting, score reporting, and measurement modeling) Prepare and review exam results and performance reports for participating associations and the Professional Practice Examination Committee Administrative responsibilities: Consult with participating associations with respect to concerns they might have on all aspects of the NPPE Collaborate with partner associations to ensure that the NPPE program is meeting the needs of all participating associations across Canada Monitor online platforms for potential breaches of exam security, including unauthorized sharing of examination content Monitor the activities of organizations offering preparatory courses for APEGA examinations Evaluate opportunities for APEGA in offering preparatory courses for NPPE Monitor APEGA’s performance and service relative to the SLA’s as well as those of the other associations Oversee all aspects of NPPE practice tests, including content development, reporting, e-commerce operations, etc. Competencies, Skills & Attributes Competencies

Knowledge, Skills & Abilities

Core: Exemplifying Integrity Fostering Communication Results Orientation Service Excellence Teamwork Core Leadership: Decision Making Ensuring Accountability Inspiring Others Functional: Attention to Detail Scientific Knowledge & Expertise

Knowledge of exam proctoring models (e.g., traditional 'brick and mortar' and remote/virtual) and test security approaches Knowledge of exam accommodations best practices Knowledge of the EGP Act, Regulations, Bylaws and BOE policies Awareness of the impacts of privacy, human rights, mobility, and other legislation upon admissions activities APEGA Registration processes and statutory obligations including a thorough understanding of a wide variety of education and experience requirements of professional engineers and geoscientists Knowledge of policies and procedures of other admissions departments in engineering and geoscience regulators across Canada Knowledge of working with item authoring and banking software systems Knowledge of working with computer-based high-stakes exam delivery platforms Translating complex data and results into layperson language and accessible reports Very high attention to detail and accuracy Excellent communication skills: both verbal and written Shows a strong commitment and focus to delivering exceptional stakeholder service Sound decision making skills Ability to understand the direction/objectives of APEGA and align individual/departmental objectives accordingly Ability to prioritize work and multi-task Proven ability to effectively work independently and in a team setting Ability to self-motivate Ability to build professional and productive relationships with colleagues, clients, and stakeholders Demonstrate ability to work with confidential information Strong leadership and interpersonal skills in dealing and communicating with professionals across diverse functional areas including technical specialists, senior and executive management, boards and committees, other provincial bodies, and members and applicants Ability to communicate ideas, solutions and resolve conflicts Strong human relations including coaching, team building and motivation skills Extensive organizational and decision-making skills

Qualifications Post-secondary degree in a related field 7-10 years of relevant experience in progressively more responsible supervisory/management roles. Prior experience in a professional environment focused on oversight of service-oriented teams involved in delivering administrative services to clients/applicants or experience in an examination’s role in a regulatory environment. At least 5 years of experience working with high-stakes examinations (certification, licensure) Working experience with Canadian professional regulators such as nurses, physicians, pharmacists, dentists, accountants, lawyers, etc. would be considered an asset Working experience in the broad area of engineering/geoscience or related professional field would be considered an asset Equivalencies may be considered. Closing Date: Until successful candidate is found We acknowledge all candidates working in their area of expertise may build their knowledge, skills, and competencies in several ways. We welcome candidates from a breadth of backgrounds, who may have equivalent skills and competencies to the ones listed above. There may be APEGA positions posted that have specific regulatory requirements, and in keeping with those requirements, APEGA will follow the guidelines that we are bound by. All candidates are considered with the objective to enhance APEGA’s people and culture. If you are interested in this position, please submit your cover letter and resume directly through our company website (https://www.apega.ca/about-apega/apega-careers). Thank you for your interest in APEGA. Only those candidates selected for interviews will be contacted.