GetYardstick Testing and Training Experts 2012-05-16T09:12:51-06:00 tag:getyardstick.com,2011:1 Copyright (c) 2011, GetYardstick Blueprinting: Determining the Weighting Formula tag:getyardstick.com,2012-05-16:/archives/20120516091231 2012-05-16T09:12:51-06:00 2012-05-16T09:00:00-06:00 Tracey Hill Last week we began a blog series based on the first stage in the Assessment Life Cycle, Defining Target and Creating Specifications. This is our third post in this series and will focus on Blueprinting: Determining the Weighting Formula. CLICK HERE to read our most recent post on this topic, Surveying Practitioners.

]]>

Last week we began a blog series based on the first stage in the Assessment Life Cycle, Defining Target and Creating Specifications. This is our third post in this series and will focus on Blueprinting: Determining the Weighting Formula. CLICK HERE to read our most recent post on this topic, Surveying Practitioners.

As mentioned in the previous article on surveying practitioners, it makes sense in some professions to ask multiple questions in the practitioner survey; frequency and criticality were mentioned as common for health professions. In this case, an additional step is required to decide how to weight the practitioner ratings from each question in the survey for calculating importance. In effect, this step determines which question is more important, frequency or criticality, and how MUCH MORE important one is than the other.

A sample process of determining these weightings involves a group of key stakeholders / policy makers. This group has been designated as the Examination Policy Group in the diagram shown below.

This group is tasked with two decisions, first, to determine which ‘dimension’ (e.g., frequency or criticality) should be more heavily weighted in the calculation of importance. Second, once one has been identified, the group is asked to select which weighting makes the most since, given the professional judgment of each of the participants. These decisions can be made either through anonymous survey or by consensus. Once the weighting formula has been decided upon, determining weights for higher levels of the competency profile can begin.

The next step is calculating survey-based competency weights which we will review in the next article. CLICK HERE to learn more about Yardstick’s psychometric services!

]]>
Blueprinting: Surveying Practitioners tag:getyardstick.com,2012-05-09:/archives/20120509130735 2012-05-10T14:11:10-06:00 2012-05-10T14:10:00-06:00 Tracey Hill This week we begin a blog series based on the first stage in the Assessment Life Cycle, Defining Target and Creating Specifications. This is our second post in this series and will focus on Blueprinting: Surveying Practitioners. CLICK HERE to read our first post, Establishing Competency Profiles.

]]>

This week we begin a blog series based on the first stage in the Assessment Life Cycle, Defining Target and Creating Specifications. This is our second post in this series and will focus on Blueprinting: Surveying Practitioners. CLICK HERE to read our first post, Establishing Competency Profiles.

Once a competency profile has been established, a representative group of Practitioners would be asked to take a survey to rate in various ways each competency. The goal of this survey is to arrive at a defensible rating of importance for each competency. Because importance is defined differently depending on profession, there are different approaches regarding what ratings the Practitioner group would provide in the survey.

For many health professions, frequency (how frequency the area is performed on the job) and criticality (the severity of consequences arising from the incorrect performance of the competency) are often used. For other professions and contexts, it is often sufficient simply to ask practitioners how important each competency is. An example of such a survey is provided below:

Obviously not everything in a competency profile is testable via a standard multiple-choice exam and so the survey, and other processes in this stage, helps identify which competency areas are testable and which are not. Performance based competencies that cannot be properly assessed via multiple-choice format exams may be testing using an OSCE (Objective Structured Clinical Examination) or other performance based model instead.

The next step is determining the weighting formula for the blueprint which we will tackle in our next article. CLICK HERE to learn more about Yardstick’s psychometric services!

]]>
Establishing Competency Profiles tag:getyardstick.com,2012-05-08:/archives/20120508090014 2012-05-09T13:09:22-06:00 2012-05-08T09:00:00-06:00 Tracey Hill The first stage in the Assessment Life Cycle, Defining Target and Creating Specifications, is where any assessment program starts. Here test developers decide what they’d like their assessment to be able to conclude about test takers, and finish by creating a crystal clear blueprint on the number and types of test questions that need to be written.

]]>

This week we begin a blog series based on the first stage in the Assessment Life Cycle, Defining Target and Creating Specifications. This is our first post in this series and will focus on Establishing Competency Profiles.

The first stage in the Assessment Life Cycle, Defining Target and Creating Specifications, is where any assessment program starts. Here test developers decide what they’d like their assessment to be able to conclude about test takers, and finish by creating a crystal clear blueprint on the number and types of test questions that need to be written.

Though we like to refer to it as one stage (Stage One), there are really two discrete (and involved!) steps in getting from the idea of an exam to having a workable blueprint:

  1. Compilation of an inventory of skills and knowledge required for mastery of the subject area (e.g., competency profile)
  2. Generation of the specifications (e.g., exam blueprint)

We will discuss the first step below, establishing a competency profile.

The compilation of an inventory of skills and knowledge is a time consuming and detailed oriented process that is vital to the success of an assessment program. The competency profile (also referred to as Competency Models, Competency Profiles, Job Maps or Job Matrices) for a profession defines what professionals should know and can do within their job role. Often, a competency profile will not only specify which skills, knowledge, behaviors, or attitudes are required for mastery of the subject area, but also the importance and/or frequency of the competency areas.

An example of a competency profile is that posted by the College of Licensed Practical Nurses of Alberta (CLPNA). The introductory paragraphs of the competency profile provide an excellent summary of the content of the competency profile and how the competency profile should be used and interpreted:

This Competency Profile includes the knowledge, skills, behaviours and attitudes required by a Licensed Practical Nurse (LPN) in Alberta as outlined in the Health Professions Act LPN Regulation 2003, and the CLPNA Standards of Practice. No one LPN is required to possess all competencies described in this document. Each LPN will possess a set of competencies specific to him/her and his/her workplace, setting or practice. In addition, the degree of proficiency of each competency will vary with different LPNs.

The competencies defined in the profile may be attained by LPNs in many different ways. Some competencies will be acquired through formal education (basic). Other competencies may be gained through experience and/or on-the-job training (additional). Some competencies are gained through advanced training, education and/or certification (specialty). Competencies seen to be “advanced” and gained specifically through additional training or specialty education are identified in the profile by shading.

A competency profile is generally broken down by high-level domain area with subdomains defined. In the case of the LPN competency profile a high-level domain area is “Nursing Knowledge” with sub-domain areas including “Anatomy and Physiology”, “Microbiology”, etc. Above is an excerpt of the LPN competency profile chart:

A competency profile can have 30 or more high-level areas with anywhere from 2 to 20 or more sub-domain areas under each. It is not uncommon for a competency profile to be hundreds of pages in length with a great deal of detail included for each sub-domain area. An example from the LPN competency profile of the details included in a sub-domain area “Anatomy and Physiology” is included below:

The next step in the process involves surveying practitioners, stay tuned for more on this process and CLICK HERE to learn more about Yardstick’s psychometric services!

]]>
Only 5 Seats Remaining for the Yardstick Seminar Psychometrics 101: Defensibility Bootcamp! tag:getyardstick.com,2012-05-01:/archives/20120501122817 2012-05-02T13:13:43-06:00 2012-05-02T13:00:00-06:00 Tracey Hill The Yardstick Psychometrics 101: Defensibility Boot Camp has received a fantastic response and seats for this exciting event are nearly sold out!

Regulators, test developers, program managers, and educators from across North America will be heading to Toronto on May 29 to take part in this comprehensive, one-day seminar.

]]>

The Yardstick Psychometrics 101: Defensibility Boot Camp has received a fantastic response and seats for this exciting event are nearly sold out!

Regulators, test developers, program managers, and educators from across North America will be heading to Toronto on May 29 to take part in this comprehensive, one-day seminar.

Registration has been fast and furious and with 35 attendees already registered, we have only 5 seats remaining!

ONLY 5 SEATS REMAINING! REGISTER TODAY

WHERE, WHEN, & WHAT
Hosted by our own expert psychometricians Greg Sadesky and Greg Pope, attendees will be given all the tools they need to demystify psychometrics and understand fully the Assessment Life Cycle.

May 29, 2012
8:30 – 5:00 PM
Toronto Renaissance Hotel Downtown
(host hotel of 2012 CME Congress)

This in-person one-day seminar will provide a comprehensive overview of all psychometric processes involved in a high stakes certification/licensure assessment context represented by the Assessment Life Cycle. You will understand each of the six stages of the Assessment Life Cycle through instruction and practical, applied, hands-on group work:

Stage 1: Defining purpose, targets, and creating specifications (competency profiles and blueprinting)
Stage 2: Filling your item bank (item development)
Stage 3: Assembling your test (test development)
Stage 4: Test administration (secure testing and standards)
Stage 5: Item and test analysis (testing psychometric quality of items)
Stage 6: Standard setting (establishing cut scores)

INCLUDES

  • One day in-person session with two industry expert psychometricians
  • Printed bound copy of the Assessment Life Cycle white paper series plus custom CD with electronic copies
  • Continental breakfast, working lunch, and refreshments

WHO SHOULD ATTEND

  • Regulators
  • Associations
  • Test Developers
  • Credentialing Boards
  • Educators
  • Testing Program Executives
  • Marketing & Production Managers in Testing

OUR EXPERTS

Greg Sadesky, Ph.D.
As VP Psychometric Services and the big man on campus for our Professional Testing Services, Greg applies his expertise to ensure that your tests are fair, sound, and interpretable. He works with our partners to demystify the process at any stage of the assessment cycle.

Greg Pope, MSc.
As VP Examination Services Greg is obsessed with ensuring that our high stakes customers get the services they need for their examination program, on time, and of the highest quality. Greg Pope and Greg Sadesky are joined at the hip when it comes to psychometrics, providing high stakes customers with soup to nuts psychometric services framed within the Assessment Life Cycle.

HOW TO REGISTER
Download the registration form and fax or mail in to the Yardstick office. Once your registration is received, we will confirm by telephone and provide you with a receipt for payment.

HAVE QUESTIONS?
If you have questions regarding this exciting event and would like to learn more, send us an email: Psychometrics@GetYardstick.com.

]]>
We're Hiring for Yardstick EAST! tag:getyardstick.com,2012-05-01:/archives/20120501105404 2012-05-01T10:55:30-06:00 2012-05-01T10:00:00-06:00 Tracey Hill We are over-the-moon excited to officially announce that Yardstick will be opening a second Canadian office and we need some great people to help run the office!

Apply today and become the next Yardsticker!

]]>

We are over-the-moon excited to officially announce that Yardstick will be opening a second Canadian office!

Located in Toronto, this new office will be our first physical expansion and will serve as the Eastern headquarters for all of our professional testing and training services. Yardstick EAST will also be home to a world-class testing centre, similar to the Edmonton Exam Centre.

Work for Yardstick EAST

Yardstick Software is a full-service solution provider for organizations delivering online testing and training. We work with Associations, Licensure and Regulatory organizations, Government and Educators to set the standards of their separate industries through licensing and certification, training, and testing.

We are searching high and low for talented individuals to join us as we expand into Toronto.  Self-driven, entrepreneurial, and positive with a strong work ethic…if those characteristics describe you, you might just be the next Yardsticker!  Take a look at our openings below and apply today.

Business Development Manager

Full-time, Permanent
Toronto, Ontario
Salary: $70,000-$110,000

The Business Development Manager will focus on opening doors in industry verticals that Yardstick currently has a known presence in, as well as introducing Yardstick as Canada’s premiere testing and training service company to regulatory organizations locally in Toronto, in the province of Ontario, and across Canada.

DOWNLOAD the full job description…

Psychometrician

Full-time, Permanent
Toronto, Ontario
Salary:  Competitive, Dependent on experience

The Psychometrician will collaborate with Yardstick staff including analysts, software developers, sales, customer support, test centre staff, and fellow psychometricians to provide high quality psychometric services to Yardstick’s high stakes testing clientele. The successful candidate would take lead on providing psychometric services that could include standard setting, item analysis, blueprinting. 

DOWNLOAD the full job description…

Test Centre Coordinator

Full-time, Permanent
Toronto, Ontario
Salary:  $35,000-$40,000

The Test Centre Coordinator is responsible for administrative tasks (e.g., proctoring, booking/confirming exams, etc.) associated with test administration services for Yardstick’s high stakes clients and the smooth operation of Yardstick’s international exam centre network. The Test Centre Coordinator is also responsible, under the supervision/direction of the Test Centre Manager, for pulling together data for regular test centre reporting and conducting content/data accuracy entry and validation for content on the Yardstick platform.

DOWNLOAD the full job description…

]]>
Yardstick Expands to Toronto tag:getyardstick.com,2012-04-30:/archives/20120430102549 2012-05-01T09:02:44-06:00 2012-05-01T08:00:00-06:00 Tracey Hill We are over-the-moon excited to officially announce that Yardstick will be opening a second Canadian office!

Located in Toronto, this new office will be our first physical expansion and will serve as the Eastern headquarters for all of our professional testing and training services. Yardstick EAST will also be home to a world-class testing centre, similar to the Edmonton Exam Centre.

]]>

We are over-the-moon excited to officially announce that Yardstick will be opening a second Canadian office!

Located in Toronto, this new office will be our first physical expansion and will serve as the Eastern headquarters for all of our professional testing and training services. Yardstick EAST will also be home to a world-class testing centre, similar to the Edmonton Exam Centre.

Expanding to Toronto will allow us to cement our existing customer relationships, while also offering the opportunity to create additional growth and advance our customer base.

“Two years ago we decided to execute a business development strategy focused delivering psychometric testing services and our award winning software platfrom to regulatory and licensing organizations. The success of that strategy has brought us to this point and we are excited to expand into Toronto as a natural progression of our growth and development,” says Chris LaBossiere, Co-Founder and Manager of Business Development.

Yardstick EAST will open in September 2012 and will be manned by expert Yardstick staff, including a high stakes testing coordinator and professional psychometrician. Included at the Toronto facility will be a fully secure testing centre, which will be added to the over 60 locations across Canada Yardstick offers it’s clients.

READ the press release…

Learn More about Yardstick EAST

Visit the Yardstick EAST page on our website to learn about the expansion and stay-up-to-date with our office progress.

Work for Yardstick EAST

Yardstick Software is a full-service solution provider for organizations delivering online testing and training. We work with Associations, Licensure and Regulatory organizations, Government and Educators to set the standards of their separate industries through licensing and certification, training, and testing.

We are searching high and low for talented individuals to join us as we expand into Toronto.  Self-driven, entrepreneurial, and positive with a strong work ethic…if those characteristics describe you, you might just be the next Yardsticker!  Take a look at our openings below and apply today.

Business Development Manager

Full-time, Permanent
Toronto, Ontario
Salary: $70,000-$110,000

The Business Development Manager will focus on opening doors in industry verticals that Yardstick currently has a known presence in, as well as introducing Yardstick as Canada’s premiere testing and training service company to regulatory organizations locally in Toronto, in the province of Ontario, and across Canada.

DOWNLOAD the full job description…

Psychometrician

Full-time, Permanent
Toronto, Ontario
Salary:  Competitive, Dependent on experience

The Psychometrician will collaborate with Yardstick staff including analysts, software developers, sales, customer support, test centre staff, and fellow psychometricians to provide high quality psychometric services to Yardstick’s high stakes testing clientele. The successful candidate would take lead on providing psychometric services that could include standard setting, item analysis, blueprinting. 

DOWNLOAD the full job description…

Test Centre Coordinator

Full-time, Permanent
Toronto, Ontario
Salary:  $35,000-$40,000

The Test Centre Coordinator is responsible for administrative tasks (e.g., proctoring, booking/confirming exams, etc.) associated with test administration services for Yardstick’s high stakes clients and the smooth operation of Yardstick’s international exam centre network. The Test Centre Coordinator is also responsible, under the supervision/direction of the Test Centre Manager, for pulling together data for regular test centre reporting and conducting content/data accuracy entry and validation for content on the Yardstick platform.

DOWNLOAD the full job description…

]]>
Bookmark Standard Setting tag:getyardstick.com,2012-04-17:/archives/20120417114222 2012-04-17T11:53:57-06:00 2012-04-17T11:00:00-06:00 Tracey Hill Last week we posted a blog discussing the fun and fabulous Angoff standard setting method. This time around we will talk about the Angoff Bookmark method.

The Bookmark method is a somewhat more intuitive method of standards setting compared to the Angoff method and is appropriate for both fixed exam forms as well as item banks using in Computerized Adaptive Testing (CAT).

]]>
Last week we posted a blog discussing the fun and fabulous Angoff standard setting method. This time around we will talk about the Bookmark method.

The Bookmark method is a somewhat more intuitive method of standards setting compared to the Angoff method and is appropriate for both fixed exam forms as well as item banks using in Computerized Adaptive Testing (CAT). The Bookmark method process involves ordering all the items on the exam form or in the item bank in terms of the item difficulty (i.e., CTT p-value or IRT b-parameter) from least difficult to most difficult.

Participants in the bookmark method go through each question and first describe the knowledge or skill that is required to answer the question correctly and also what additional knowledge beyond the previous, easier question is required. Then, each panelist makes a judgment regarding whether a borderline competent candidate would get the question correct or not. As the standard setting participant goes through the items, as the items get harder and harder, the participant will come closer to the “uncomfortable line” where a borderline acceptable candidate would be able to get the item correct and where that borderline acceptable candidate would get the item wrong. Standard setting participants show their judgment of the cut-score by placing a “bookmark” between the items where the cut-score should fall.

An example of this process for a standard setting participant is shown below.

Each standard setting participant establishes their bookmark and several rounds of discussion regarding the discrepancies between participant judgments occur. In the example above, one standard setting participant placed their cut-score judgment between item 4 and 5, another standard setting participant may place their bookmark between items 5 and 6. Through facilitated group discussion and review the differences between raters are discussed in terms of the knowledge and skill that candidates and students ought to have, and therefore what justified the placement of the bookmark.

As a result of this discussion, greater consensus is typically reached on the position of the bookmarks. Actual data from candidates also is often provided as impact data, much in the same way and purpose as for the Angoff procedure. Once all the rounds have been completed the final cut score is calculated taking into account the bookmark judgments of all the standard setting participants. The Examination Policy Group, prior to being implemented for candidates, generally vets the proposed cut score.

Learn more about this method and all of the psychometric services offered by the Yardstick team.

]]>
Yardstick April Psychometric Newsletter tag:getyardstick.com,2012-04-16:/archives/20120416130230 2012-04-16T13:34:46-06:00 2012-04-16T13:00:00-06:00 Tracey Hill Download the Yardstick April Psychometrics newsletter and learn about our participation at the 2012 CME Congress in Toronto, the Angoff Standard Setting Approach, and an innovative Yardstick client that is taking advantage of live proctors and screen sharing to expand their existing market base.

]]>
Download the Yardstick April Psychometrics newsletter and learn about our participation at the 2012 CME Congress in Toronto, the Angoff Standard Setting Approach, and an innovative Yardstick client that is taking advantage of live proctors and screen sharing to expand their existing market base.

Click HERE to read the entire newsletter and learn more about Yardstick’s testing and training services.

]]>
The Early Bird Gets the Psychometric Worm tag:getyardstick.com,2012-04-11:/archives/20120411130425 2012-04-11T13:09:25-06:00 2012-04-11T13:00:00-06:00 Tracey Hill The Yardstick Psychometrics 101: Defensibility Boot Camp has received a fantastic response and seats for this exciting event are quickly filling up. Early bird pricing ends soon, so register today and receive a discount on your sign-up fee.

EARLY BIRD PRICING ENDS APRIL 15 AT MIDNIGHT!

]]>

The Yardstick Psychometrics 101: Defensibility Boot Camp has received a fantastic response and seats for this exciting event are quickly filling up. Early bird pricing ends soon, so register today and receive a discount on your sign-up fee.

EARLY BIRD PRICING ENDS APRIL 15 AT MIDNIGHT! REGISTER TODAY

WHERE, WHEN, & WHAT
Hosted by our own expert psychometricians Greg Sadesky and Greg Pope, attendees will be given all the tools they need to demystify psychometrics and understand fully the Assessment Life Cycle.

May 29, 2012
8:30 – 5:00 PM
Toronto Renaissance Hotel Downtown
(host hotel of 2012 CME Congress)

This in-person one-day seminar will provide a comprehensive overview of all psychometric processes involved in a high stakes certification/licensure assessment context represented by the Assessment Life Cycle. You will understand each of the six stages of the Assessment Life Cycle through instruction and practical, applied, hands-on group work:

Stage 1: Defining purpose, targets, and creating specifications (competency profiles and blueprinting)
Stage 2: Filling your item bank (item development)
Stage 3: Assembling your test (test development)
Stage 4: Test administration (secure testing and standards)
Stage 5: Item and test analysis (testing psychometric quality of items)
Stage 6: Standard setting (establishing cut scores)

INCLUDES

  • One day in-person session with two industry expert psychometricians
  • Printed bound copy of the Assessment Life Cycle white paper series plus custom CD with electronic copies
  • Continental breakfast, working lunch, and refreshments

WHO SHOULD ATTEND

  • Regulators
  • Associations
  • Test Developers
  • Credentialing Boards
  • Educators
  • Testing Program Executives
  • Marketing & Production Managers in Testing

OUR EXPERTS

Greg Sadesky, Ph.D.
As VP Psychometric Services and the big man on campus for our Professional Testing Services, Greg applies his expertise to ensure that your tests are fair, sound, and interpretable. He works with our partners to demystify the process at any stage of the assessment cycle.

Greg Pope, MSc.
As VP Examination Services Greg is obsessed with ensuring that our high stakes customers get the services they need for their examination program, on time, and of the highest quality. Greg Pope and Greg Sadesky are joined at the hip when it comes to psychometrics, providing high stakes customers with soup to nuts psychometric services framed within the Assessment Life Cycle.

HOW TO REGISTER
Download the registration form and fax or mail in to the Yardstick office. Once your registration is received, we will confirm by telephone and provide you with a receipt for payment.

EARLY BIRD PRICING UNTIL APRIL 15! REGISTER TODAY

HAVE QUESTIONS?
If you have questions regarding this exciting event and would like to learn more, send us an email: Psychometrics@GetYardstick.com.

]]>
Yardstick Supports “2012 Hockey Marathon” tag:getyardstick.com,2012-04-10:/archives/20120410102633 2012-04-10T11:41:51-06:00 2012-04-10T10:00:00-06:00 Tracey Hill We are happy to be a donating sponsor of the 2012 Hockey Marathon for the Kids, in support of the Alberta Children’s Hospital Foundation.

This exciting and world record breaking event will be held from May 6-16 and will feature volunteers from across Alberta playing hockey for 10 days, 24 hours per day in an effort to break the current world record of 243 hours. All proceeds raised in support of the event will go to the Alberta Children’s Hospital Foundation, which will support life saving equipment for PICU and cancer research at the Alberta Children’s Hospital.

]]>

We are happy to be a donating sponsor of the 2012 Hockey Marathon for the Kids, in support of the Alberta Children’s Hospital Foundation.

This exciting and world record breaking event will be held from May 6-16 and will feature volunteers from across Alberta playing hockey for 10 days, 24 hours per day in an effort to break the current world record of 243 hours. All proceeds raised in support of the event will go to the Alberta Children’s Hospital Foundation, which will support life saving equipment for PICU and cancer research at the Alberta Children’s Hospital.

We first learned about the Hockey Marathon from Yardstick friend, Greg Britton, who will be volunteering and playing hockey in the event. Greg is a human resources training professional who has worked with leading organizations like FT Services and Winroc and who has utilized Yardstick’s T2 platform in a corporate training capacity.

We are proud to be associated with Greg and the Hockey Marathon and wish the best of luck to all the volunteers as they dedicate their efforts for this worthy cause!

For more information on the event and to learn how you can donate, visit their website.

]]>
Yardstick Hiring High Stakes Testing Coordinator - FILLED tag:getyardstick.com,2012-04-09:/archives/20120409111548 2012-05-02T13:14:13-06:00 2012-04-09T11:15:00-06:00 Tracey Hill We are looking for a dynamic, master multi-tasker, who has experience managing high stakes relationships! Are you the new Yardsticker?

Yardstick Software is a full-service solution provider for organizations delivering online testing and training. We work with Associations, Licensure and Regulatory organizations, Government, and Educators to set the standards of their separate industries through licensing and certification, training, and testing.

]]>

We are looking for a dynamic, master multi-tasker, who has experience managing high stakes relationships! Are you the new Yardsticker?

Yardstick Software is a full-service solution provider for organizations delivering online testing and training. We work with Associations, Licensure and Regulatory organizations, Government, and Educators to set the standards of their separate industries through licensing and certification, training, and testing.

High Stakes Testing Coordinator

Full-Time Position, Permanent
Annual Salary: $40,000 – $45,000
Company Benefits, 2 Weeks Annual Vacation

The High Stakes Testing Coordinator is responsible for the overall management of test administration services for Yardstick’s high stakes clients and the smooth operation of Yardstick’s international exam centre network. High stakes customers (e.g., certification/licensure testing organizations) require special care and attention that the High Stakes Testing Coordinator would co-manage. Test centre locations, where these high stakes exams are administered, must adhere to strict standards and require attention to ensure that their questions are answered and that the candidate experience is fair and as free of anxiety as possible. This position reports to the Vice President Examination Services in Edmonton.

Primary Roles and Responsibilities

  • Operational co-management (in conjunction with the Test Centre Manager) of high stakes exam accounts, including client liaison and relationship building
  • Identification and execution of test centre network requirements
  • Implementation of exam administration standards and protocols (compliance)
  • Overseeing the effective and timely execution of candidate exam centre bookings/actions
  • Development of customized proctor guides and providing proctor training
  • Expansion of exam centre network to fill client needs including contract negotiations
  • Overseeing execution of customer service calls and exam centre confirmation calls
  • Other various task from time to time, as directed by senior management or their designate

Educational Requirements, Experience, and Attributes

  • Comfortable with technology
  • Excellent project management skills
  • Enjoys working in a collaborative team environment
  • Excellent communication skills both verbally and in written form
  • High proficiency using Microsoft Office applications (Word, Excel, Outlook)
  • Calm under pressure, able to provide instruction to proctors/candidates under timelines
  • Comfortable providing online training (e.g., proctor training)
  • Very high attention to detail and precise nature essential
  • Experience working with certification/licensure testing organizations preferred
  • Experience proctoring, or overseeing the proctoring of high stakes examinations an asset

DOWNLOAD the full job description.

For more information on this opportunity or to apply for the role, please email: GreatJobs@GetYardstick.com.

]]>
Angoff Standard Setting Approach tag:getyardstick.com,2012-04-09:/archives/20120409092919 2012-04-09T09:33:22-06:00 2012-04-09T09:00:00-06:00 Tracey Hill Standard Setting, the business of setting passing scores, (AKA ‘cut scores’), for examinations. The cut score(s) serves to classify candidates into categories; those who score above the cut score are judged to possess the minimum level of proficiency required for inclusion into the category, while those who score below the cut score are deemed not to possess the minimum level of proficiency required, and are classified accordingly.

]]>

Standard Setting is the business of setting passing scores, (AKA ‘cut scores’), for examinations. The cut score(s) serves to classify candidates into categories; those who score above the cut score are judged to possess the minimum level of proficiency required for inclusion into the category, while those who score below the cut score are deemed not to possess the minimum level of proficiency required, and are classified accordingly.

Standard Setting is most often a judgmental process. It typically involves a panel of subject matter experts and stakeholders who must estimate the difficulty of each question for so-called minimally-competent, or borderline candidates. Those judgments are then aggregated in order to arrive at a passing score across sets of questions. Standard setting methods differ in the way that those judgments are made by panelists, and also in the way that question-level judgments are aggregated to create a passing score. We will talk about one of the most common methods of standard setting, the Modified Angoff method and save the Bookmark method for another time.

The central task of the Modified Angoff method is for panelists to estimate the percentage of minimally competent candidates who would answer each item correctly. The instructions to panelists would be to examine the question carefully, both in terms of the ‘structure’ of the question, and of the difficulty of the competency being tested. This information is used to make a judgment regarding the expected performance on that question of the minimally competent candidate. Two types of judgments are common, either the probability that any single candidate would answer the question correctly, or the number out of 100 minimally competent candidates that would answer the question correctly.

These judgments are summed for each panelist to create a recommendation for a passing score. This makes sense, because the item-level judgments are themselves pass-level judgments per question. For example, if a panelist provided judgments of 0.7, or 70% for each and every question, the passing standard would logically be 70% for the entire exam.

Typically these judgments are made over multiple rounds after which the judgments become increasingly refined. Between rounds, various types of information can be provided to panelists regarding the reasonableness of their judgments. A common type of information is impact data, or the number and percentages of candidates who would passed based upon the average or median passing score recommendation by panelists. Other types of feedback include the difficulty of each question for the candidates or the consistency of each panelist’s judgments.

As shown in the figure below, the changes in item-level ratings impact the overall recommended passing score. The passing score recommendation made in the final round is the one that is ultimately adopted.

For more detailed information on standard setting procedures and methods see the book, Setting Performance Standards: Concepts, Methods, and Perspectives, edited by Gregory Cizek and Robert Sternberg.

Learn more about this method and all of the psychometric services offered by the Yardstick team.

]]>
What kind of folks work at Yardstick? tag:getyardstick.com,2012-04-04:/archives/20120404090032 2012-04-04T09:06:25-06:00 2012-04-04T09:00:00-06:00 Tracey Hill Ever wonder what type of folks work at Yardstick? Well, one of Yardstick’s Core Values is:

Attract and Empower Great People

…and when we say great, we mean GREAT! Case in point, our new Sales and Operations Coordinator, Cindi Cunningham. Cindi is a talented administrator, a fun-loving family girl, with an easy ability to poke fun at herself.

]]>

Ever wonder what type of folks work at Yardstick? Well, one of Yardstick’s Core Values is:

Attract and Empower Great People

…and when we say great, we mean GREAT!

Case in point, our new Sales and Operations Coordinator, Cindi Cunningham. Cindi is a talented administrator, a fun-loving family girl, with an easy ability to poke fun at herself.

Last year, Cindi had an opportunity to be featured on the Ellen Show in a segment called “Can you believe these bad paid-for photos?” If you aren’t familiar with Ellen Degeneres, she has a syndicated talk show based in Hollywood that includes a segment that highlights crazy photos of Ellen viewers. This is where Cindi comes in.

Cindi’s photo is described as “my grade 11 school picture at a rather conservative private school” and it’s one of the funniest photos we’ve ever seen. Check out the first 30 seconds of the video below and see why we are proud to have Cindi as part of the Yardstick team.

]]>
Yardstick Chosen to Present at 2012 CME Congress tag:getyardstick.com,2012-03-27:/archives/20120327122936 2012-03-27T12:43:21-06:00 2012-03-27T12:00:00-06:00 Tracey Hill The 2012 CME Congress is upcoming in May and we are thrilled to announce that Yardstick has been chosen as poster session presenters at the event.

This prestigious event is recognized as the preeminent international conference for researchers and practitioners in CME/CPD in the health professions. Hosted by the University of Toronto and focused on Innovation in Learning and Assessment of Health Professionals, the Congress is a critical networking
conference for CME/CPD professionals, with over 500 international attendees.

]]>

The 2012 CME Congress is upcoming in May and we are thrilled to announce that Yardstick has been chosen as presenters at the event.

This prestigious event is recognized as the preeminent international conference for researchers and practitioners in CME/CPD in the health professions. Hosted by the University of Toronto and focused on Innovation in Learning and Assessment of Health Professionals, the Congress is a critical networking conference for CME/CPD professionals, with over 500 international attendees.

Yardstick will be hosting three presentation sessions, as follows:

Doing it Right: The Promise of Diagnostic Reporting in a CE Context
Greg Sadesky, VP Psychometric Services
& Greg Pope, VP Examination Services

Reporting of information is both an art and a science. In a continuing education context, a well-designed report could mean the difference between a well-executed professional development plan and one without clear and defensible focus. This is particularly true when such a plan depends upon the feedback from the performance on a formative assessment tool, such as a Knowledge Assessment. Increasingly, many health professions are including as a component of continuing education, an assessment that provides information on the performance of the professional in terms of competency areas. In order to maximize the potential of the assessment in terms of professional development, it would be extremely beneficial to draw on best practice guidelines in the area of information visualization. These guidelines would be used to provide test-takers with reports that are informative, yet intuitive and interpretable.

Using mainly examples from a computer-based, online context, this session provides a journey through the current and future landscape of diagnostic reporting that helps realize the potential of using assessment data to create targeted and meaningful professional development plans. Session attendees will take from this session reporting principles, ideas, and models that can be used to enhance their continuing education programs.

Innovative Online Testing and Training Solutions in a CE Context
Greg Kureluk, VP Sales and Marketing

Health organizations require testing and training software platforms that are flexible and intuitive. The ability to provide courses and testing, both for formative learning as well as high stakes jurisprudence and knowledge assessment, is crucial. A system that allows for item development to occur online from anywhere in the world can create virtual collaboration and cost reduction. Item bank administrators must have at their fingertips all the reports and configuration options required to manage the item development and banking processes. Offering courses, quizzes, test, and even high stakes proctored exams on the same platform provide a plethora of flexible options for organizations. Add to this the creative application of e-commerce solutions in continuing education testing and training contexts. From the re-sale of retired exam content for practice testing, to selling online continuing education, to managing revenue streams via e-commerce.

This session will showcase modern advancements in item development, banking, and test administration including visual, versatile, vivacious software that live up to the promise of unsurpassed scalability and flexibility.

CE Jurisprudence Assessment for LPNs: A Computer Based Strategy
Greg Sadesky, VP Psychometric Services
& Greg Pope, VP Examination Services

(with Tamara Richter, Director of Operations for the CLPNA)

Jurisprudence examinations are used in a variety of contexts and for a variety of purposes among the health professions in Canada. On the one hand, some of the regulators of these professions choose to administer jurisprudence exams under a high-stakes model, and the performance on these exams have significant consequences for the candidates’ ability to practice. Others choose more of a formative approach in which the exam is viewed more as an opportunity to focus professional development. The College of Licensed Practical Nurses of Alberta (CLPNA), in partnership with Yardstick, have devised an innovative approach to conducting jurisprudence assessment under a formative model, utilizing the latest in computer based test development and administration services. The requirements of the project were unique and multi-faceted: Low cost, high flexibility for candidates and administrators, rapid turn-around of items and exam forms, impeccable psychometric defensibility, non-proctored test administration, and leading edge formative reporting back to candidates.

This session will present details of the computer based online assessment solution that was designed, developed, and implemented for the CLPNA. It will also highlight successes and challenges during this process, focusing on how a streamlined yet highly defensible online assessment instrument came to pass. Attendees will learn not only about the experience of the CLPNA in putting the Jurisprudence Exam project together, but also about how to apply a conceptual framework, the Assessment Life Cycle, to engineer examination defensibility in low, medium, or high stakes contexts.

For more information on our participation at the 2012 CME Congress, email us at: psychometrics@GetYardstick.com.


The Yardstick Gregs

]]>
Yardstick Partner Hosts Association Workshop tag:getyardstick.com,2012-03-21:/archives/20120321131818 2012-03-21T13:22:08-06:00 2012-03-21T13:00:00-06:00 Tracey Hill In May, the Alberta Mortgage Brokers Association will be hosting a one-day workshop focussed on helping today’s associations move their thinking and approaches ‘out of the box’ to keep and grow their membership.

We are happy to be sponsoring this informative event, which will be held in Calgary on May 9, 2012. AMBA is a Yardstick partner that uses our platform to deliver mortgage broker licensing and certification training and testing for their Alberta-based members.

]]>

In May, the Alberta Mortgage Brokers Association will be hosting a one-day workshop focussed on helping today’s associations move their thinking and approaches ‘out of the box’ to keep and grow their membership.

We are happy to be sponsoring this informative event, which will be held in Calgary on May 9, 2012.

AMBA is a Yardstick partner that uses our platform to deliver mortgage broker training for their Alberta-based members.

To learn more about this event, visit their website or contact them at: 1-888-452-2652.

]]>
Yardstick Partner Expands Exam into the Global Market tag:getyardstick.com,2012-03-15:/archives/20120315102804 2012-03-16T09:48:06-06:00 2012-03-16T08:40:00-06:00 Tracey Hill A few months back we announced our new partnership with ProctorU. ProctorU provides live proctors via web cams and screen sharing in order to invigilate examinations anywhere in the world and our partnership with them means we can offer cost effective virtual/remote proctoring services to Yardstick clients.

]]>
You may remember a few months back we announced a new partnership with ProctorU. ProctorU provides live proctors via web cams and screen sharing in order to invigilate examinations anywhere in the world and our partnership with them means we can offer cost effective virtual/remote proctoring services to Yardstick clients.

One of Yardstick’s first clients to take advantage of this new exposure opportunity is the team at the Institute for Safety and Health Management. Dedicated to safety in all environments, the ISHM is the credentialing organization founded by the National Safety Management Society (NSMS), to promote the establishment of professional standards.

These guys take safety seriously and we are excited to extend to them the opportunity to offer the CSHM exam virtually anywhere there is an acceptable internet connection. Tapping into our exam centre network and working with the services offered by ProctorU, ISHM and Yardstick can now deliver exams anywhere in the world…a valuable resource for international companies that want to ensure safety amongst their teams.

Tom Slavin, Chair of the ISHM Exam Committee says, “As EHS managers deal with a global economy it becomes clear that the focus of safety and health management must be broader than any one county’s laws or culture. It is great that the CSHM will be able to recognize those who have mastered the practice of safety and health management throughout the world. “

We also talked to the ISHM about their experience connecting with the new ProctorU service:

1. How will ISHM candidates benefit from the availability of global exam sittings?

ISHM is experiencing increased interest in our certification globally. Until recently we were unable to offer the certification exam to international candidates. The testing capability will certainly increase our customer base.

2. How easy was it to get set up?

The set up was seamless for ISHM and we were happy to work with Yardstick in this endeavour. The team at Yardstick was able to set up our ProctorU connection easily and in a timely fashion.

3. How do you envision it will grow your business?

We expect a significant increase in international business as our customers discover this new opportunity.

4. How has your partnership with Yardstick evolved over the last three years?

We initially began working with Yardstick in 2009 to deliver our exams online. Since then, we have utilized other services including their exam centre network in the United States and their psychometric services as well. We appreciate the Yardstick expertise in online testing and high-stakes testing. Now with our expansion into the global market using ProctorU, it makes us even happier with our partnership with Yardstick.

The collaboration between the ISHM, Yardstick, and ProctorU is another great example of technology servicing the needs across a global economy.

Click here for more information on the Yardstick platform, our exam centre network, and the services offered by Proctor U.

]]>
Yardstick March Psychometrics Newsletter tag:getyardstick.com,2012-03-15:/archives/20120315094639 2012-03-15T09:46:53-06:00 2012-03-15T09:00:00-06:00 Tracey Hill Download the Yardstick March Psychometrics newsletter and learn about our work with the College of Licensed Practical Nurses of Alberta (CLPNA), how we showed at the ATP Innovations in Testing conference, and how and where our exam centre network is growing.

]]>
Download the Yardstick March Psychometrics newsletter and learn about our work with the College of Licensed Practical Nurses of Alberta (CLPNA), how we showed at the ATP Innovations in Testing conference, and how and where our exam centre network is growing.

Click HERE to read the entire newsletter and learn more about Yardstick’s testing and training services.

]]>
Yardstick Exam Centre Network Grows tag:getyardstick.com,2012-03-13:/archives/20120313105113 2012-03-16T15:45:03-06:00 2012-03-13T10:00:00-06:00 Tracey Hill The Yardstick exam centre network encompasses hundreds of locations and currently delivers thousands of exams per year internationally. What started as seven or eight Alberta-based exam centre locations six years ago has grown into a multi-country, international network that offers state-of-the-art testing innovation and world-class proctoring professionals.

]]>

The Yardstick exam centre network encompasses hundreds of locations and currently delivers thousands of exams per year internationally.

What started as seven or eight Alberta-based exam centre locations six years ago has grown into a multi-country, international network that offers state-of-the-art testing innovation and world-class proctoring professionals.

Whether it is paper-based testing or our speciality computer based testing, we work with our proctors to ensure a smooth and successful delivery each and every time. Our proctors and test centres adhere to international best practice standards in proctored test centre delivery. Yardstick is a member of the National College Testing Association and ensures that all test centre locations follow the NCTA Professional Standards and Guidelines for Post-Secondary Test Centres.

We continue to grow our exam centre network and are excited to announce new partnerships with some prestigious organizations:

University of Western Ontario
Located in London, Ontario, the University of Western Ontario is a full-campus university offering students access to world-class learning opportunities.

Memorial University
Memorial University has campus locations across Newfoundland and Labrador, with its largest campus located in St. John’s. As the largest university in the region, this location hosts over 18,000 students annually in full and part-time studies.

Selkirk College
With eight campuses across southern British Columbia, Selkirk College is regional community college that is dedicated to inspiring, engaging, and enabling learners to be valuable contributors to their communities and to society as a whole.

College of the Rockies
With a main campus in Cranbrook and regional campuses in Creston, Fernie, Golden, and Invermere, the College of the Rockies offers a professional educational environment in a small, personalized learning experience.

British Columbia Institute of Technology
BCIT operates campuses throughout central Vancouver and offers exam candidates a professional exam-taking environment. BCIT is dedicated to small class sizes and an innovative technology environment.

University of Western Ontario
Located in London, Ontario, the University of Western Ontario is a full-campus university offering students access to world-class learning opportunities.

Memorial University
Memorial University has campus locations across Newfoundland and Labrador, with its largest campus located in St. John’s. As the largest university in the region, this location hosts over 18,000 students annually in full and part-time studies.

Click here to learn more about these new exam centre locations and all of our international locations.

Want more…
Exam scheduling, event booking, item bank management, generating exam forms, Automated Test Assembly (ATA), exam delivery strategy, conducting Classical Test Theory or Item Response Theory based item and test analysis, and event managing every avenue in-between…everything you need is available from our psychometric and test professionals.

From start to finish, Yardstick collaborates with you to ensure smooth delivery and execution of your certification and licensure program. Find out more…

]]>
Noted Psychologist Shawn Achor Chooses Yardstick to Deliver Happiness Training at PositiveResearch.com tag:getyardstick.com,2012-03-05:/archives/20120305132556 2012-03-07T08:33:37-07:00 2012-03-07T08:00:00-07:00 Tracey Hill The Yardstick platform supports many different types of training delivery. Daily we work with clients that deliver employee training, association member training, and even clients who are selling e-learning in a retail environment.

]]>

The Yardstick platform supports many different types of training delivery. Daily we work with clients that deliver employee training, association member training, and even clients who are selling e-learning in a retail environment.

One of those customers is the Institute for Applied Positive Research. Founded by noted author and psychologist Shawn Achor and his team, PostitiveResearch.com offers training programs designed to use the principle of positive psychology to raise the bar of happiness and success.

We talked to the Institute about their experience working with Yardstick and here’s what they say:

Why did you choose Yardstick?
We liked the built-in e-commerce system, the advanced reporting features, affiliate program, and the opportunity to build white label site for corporate partners.

Talk about your experience with the Yardstick team.
Yardstick has been easy to work with, from site design to execution. The platform is solid and our users appreciate how easy it is to contact support if any issues arise. Yardstick makes the process of bringing a course to market a snap.

What were some of the challenges in launching your site?
We found that the project design took longer than anticipated and we discovered that the course execution encountered some glitches, which was unexpected. We also hope to see some advancement in the revenue disbursement of the affiliate program with respect to payment directly through the system.

You launched two months ago…how do you feel about the direction of your e-learning business?
We are really excited with the increased professionalism of our e-learning business. Sales have skyrocketed and we only envision them going up from here.

The Institute currently offers two online training courses using the Yardstick platform:

The Happiness AdvantageTM eCourse is a full semester of content on the science of happiness and success.

The Happiness Resolution eCourse is a shortened version of the full Happiness Advantage eCourse which focuses on core skills needed to tackle new projects, new habits, and new resolutions.

Both courses are the work of Shawn Achor and we are excited to be partnering with him to deliver these innovative programs.

Shawn is the winner of over a dozen distinguished teaching awards at Harvard University, where he delivered lectures on positive psychology. Known for his humorous take on society’s need to reach for more than average, Shawn presented at TEDxBloomington in 2011, giving a fast-moving and entertaining talk highlighting the concept that happiness inspires productivity.

We love this video so much, we wanted to share it with you. Take a look!

]]>
Yardstick's February Newsletter tag:getyardstick.com,2012-03-01:/archives/20120301122259 2012-03-01T12:23:47-07:00 2012-03-01T12:00:00-07:00 Tracey Hill Download the Yardstick February newsletter today and learn about our newest white paper (on Standard Setting), our recent exciting partnership with the College of Licensed Practical Nurses of Alberta (CLPNA), and our upcoming Defensibility Boot Camp.

]]>
Download the Yardstick February newsletter today and learn about our newest white paper (on Standard Setting), our recent exciting partnership with the College of Licensed Practical Nurses of Alberta (CLPNA), and our upcoming Defensibility Boot Camp.

Click HERE to read the entire newsletter and learn more about Yardstick’s testing and training services.

]]>