E-Consultation Guide - User contributions [en]

NIYF Consultation: Perceived Benefits

2008-04-14T22:06:36Z

Jjh:

A number of perceived benefits have been observed to date:

* Young people participating have expressed their enthusiasm for the discussion forum environment.
* Likewise, the Consultation coordinator commented on the usefulness of having a platform that allowed young people to continue their conversations over a longer period.

NIYF & Child Protection

2008-04-14T22:05:57Z

Jjh: /* =Registration */

===The nature of the NIYF work===
[http://www.niyf.org/ NIYF] work with young people carries with it legal and ethical responsibilities in terms of providing a safe and secure envirnoment free from the threat of harm.

Upon introducing these online methods it became quickly apparent that a careful approach would be necessary in order to ensure that these requirements were met. During observations of the face-to-face consultation processes, it became apparent that online mediums' capacity to facilitate anonymity could prove beneficial, particularly when discussing sensitive issues.

====Registration====
In particular, the discussion forum software posed a threat because, in its generic form, it allowed anyone to register and post messages. It was paramount that those registered must be young people and that they adhere to a code of conduct. The registration process was, therefore, customised to ensure that only young people could register and that they agreed to the code of conduct established. While this process caused a slight delay between registration and participation, the safety of participants was considered by far the more important issue.

====Posting====
Another generic aspect of the discussion forum also posed a threat; the fact that the forum was available 24-7 meant that it required particularly vigilant moderation. A facility was set up to alert the moderator via e-mail when a post to the forum had been made and it became necessary for the moderator to check for posts at regular intervals, including evenings and weekends, in case any inappropriate material was posted.

===Significant Challenge===
Despite a desire on the part of Smyth and [http://www.niyf.org/ NIYF] to roll out the online consultation activity to as wide an audience as possible - child protection considerations have hindered this development

It is now believed that adequate safeguards have been developed incorporating verifiable registration and vigilant moderation with any generic loopholes allowing access having been closed.

NIYF & Child Protection

2008-04-14T22:05:38Z

Jjh:

===The nature of the NIYF work===
[http://www.niyf.org/ NIYF] work with young people carries with it legal and ethical responsibilities in terms of providing a safe and secure envirnoment free from the threat of harm.

Upon introducing these online methods it became quickly apparent that a careful approach would be necessary in order to ensure that these requirements were met. During observations of the face-to-face consultation processes, it became apparent that online mediums' capacity to facilitate anonymity could prove beneficial, particularly when discussing sensitive issues.

=====Registration====
In particular, the discussion forum software posed a threat because, in its generic form, it allowed anyone to register and post messages. It was paramount that those registered must be young people and that they adhere to a code of conduct. The registration process was, therefore, customised to ensure that only young people could register and that they agreed to the code of conduct established. While this process caused a slight delay between registration and participation, the safety of participants was considered by far the more important issue.

====Posting====
Another generic aspect of the discussion forum also posed a threat; the fact that the forum was available 24-7 meant that it required particularly vigilant moderation. A facility was set up to alert the moderator via e-mail when a post to the forum had been made and it became necessary for the moderator to check for posts at regular intervals, including evenings and weekends, in case any inappropriate material was posted.

===Significant Challenge===
Despite a desire on the part of Smyth and [http://www.niyf.org/ NIYF] to roll out the online consultation activity to as wide an audience as possible - child protection considerations have hindered this development

It is now believed that adequate safeguards have been developed incorporating verifiable registration and vigilant moderation with any generic loopholes allowing access having been closed.

NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation

2008-04-14T21:57:41Z

Jjh:

===Doctoral Research Project===
Throughout the course of the ECRG Project, sponsorship has been provided for Michele Smyth to undertake research, at PhD degree level. Smyth has focused her studies in the e-consultation field. The topic for this research is "'''Exploring capacity for youth participation in online public consultation'''".

===About Northern Ireland Youth Forum (NIYF)===
The [http://www.niyf.org/ Northern Ireland Youth Forum (NIYF)] is a key agency in current youth participation activity.

For over 25 years the forum has been a central force in developing mechanisms for enabling youth participation across Northern Ireland.

Working with the 14–25 age group, [http://www.niyf.org/ NIYF] currently delivers a number of programmes that characteristically incorporate innovative techniques and a high level of interactivity. These programmes are delivered in various ‘real world’ settings such as youth clubs.

===Initiating the Research Project - Aim and Purpose===
Through discussions with [http://www.niyf.org/ NIYF], a shared research interest in e-enabled approaches to consultation, and their potential capacity to enrich current youth consultation activities, has been established. Access has been negotiated to conduct a research programme.

Smyth's research explores the capacity for e-enabling appropriate aspects of youth participation in public consultation within existing structures in Northern Ireland. Various stakeholder perspectives will be interpreted with a view to developing an online resource that has the potential to offer experiential learning opportunities on citizenship and its related themes within classroom settings.

===Project Methodology===
Research data was collected by shadowing a youth worker, directly involved in engaging young people in public consultation processes. In academia, this method is called "action research".

In keeping with the action research approach, research activity:
*observes practice,
*then, identifies and implement appropriate e-enabled interventions
*interventions and emerging issues are observed
*findings then inform the next cycle in an iterative process
This process is called cyclical enquiry.

===Project Value===
The purpose of the cyclical enquiry is to iteratively inform the design of appropriate online mechanisms that may be used by [http://www.niyf.org/ NIYF] to complement their existing consultation work.

During the consultation processes, where appropriate, an intervention in the form of an e-enabled component has been introduced. To date, a website using [http://en.wikipedia.org/wiki/Wiki Wiki-technology] has been set up to act as a platform for complementary e-consultation

===The youth e-consultation website===
Over the last 12 months a number of online technology interventions have been introduced: discussion forum software and online survey software.

The table below serves as a guide for consulting bodies approaching [http://www.niyf.org/ NIYF] to undertake a consultation on their behalf, illustrating the opportunities and constraints of the methods listed.

[[Image:NIYF_tech_table.JPG]]

This table illustrates e-consultation methods [http://www.niyf.org/ NIYF] currently offers. Note, however, that we are continually developing methods and trying out different technologies. Any of these methods can be tailored and combined to fit specific consultation requirements.

===Other Considerations===
*Promotion: Whatever the method - it is essential that it be appropriately promoted.
*Timeframes: As a general rule of thumb, an ‘activity’ period of 6-8 weeks should be accommodated within the planned timeframe. Please note this does not include recruitment, setting up & reporting after the event.
*Incentives: You may wish to provide incentives for participating, depending on budgetary constraints, as these can encourage participation

Youth e-consultations at NIYF

2008-04-14T21:44:48Z

Jjh: /* NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation */

== [[NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation]] ==

This study was undertaken by [http://www.e-consultation.org ECRG Project] sponsorsed Michele Smyth, reading for a PhD degree on "Exploring capacity for youth participation in online public consultation".

===What is the NIYF===
The [http://www.niyf.org/ Northern Ireland Youth Forum (NIYF)] is a key agency in current youth participation activity, developing mechanisms for enabling youth participation across [http://www.discoverireland.com Northern Ireland]. Working with the 14–25 age group, [http://www.niyf.org/ NIYF] currently delivers a number of programmes in various settings, such as youth clubs.

===Doctoral Research===
Smyth 's research explored the capacity for e-enabling appropriate aspects of youth participation in public consultation within existing structures in Northern Ireland. Various stakeholder perspectives were interpreted with a view to developing an online resource offering experiential learning opportunities on citizenship and related themes.

The purpose of the research is to inform the design of appropriate online mechanisms that may be used by [http://www.niyf.org/ NIYF] to complement their existing consultation work. To date, a website using wiki technology has been set up to act as a platform for complementary e-consultation activity and a number of online technology interventions have been introduced

To date, a number of issues have emerged: Recruitment and maintaining interest, and Child Protection

== [[NIYF: Respondent Recruitment]] ==
Recruiting respondents for a consultation process and maintaining interest throughout the process is a key issue. Participants need to know why it is important for them to take part. What specifically is at stake? In what way will their contributon, and its impact, count?

== [[NIYF & Child Protection]] ==
The nature of the [http://www.niyf.org/ NIYF] work with young people carries with it legal and ethical responsibilities in terms of providing a safe and secure environment free from the threat of harm. Upon introducing these online methods it became quickly apparent that a careful approach would be necessary in order to ensure that these requirements were met.

Despite a desire on the part of Smyth and [http://www.niyf.org/ NIYF] to roll out the online consultation activity to as wide an audience as possible, child protection considerations have hindered this development.

== [[NIYF Consultation: Perceived Benefits]] ==
A number of perceived benefits have been observed to date:
*Young participant expressed enthusiasm for the discussion forum environment, and,
*The Consultation coordinator commented on the usefulness of having such a platform.

Youth e-consultations at NIYF

2008-04-14T21:43:33Z

Jjh: /* NIYF & Child Protection */

== [[NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation]] ==

This study was undertaken by ECRG Project sponsorsed Michele Smyth, reading for a PhD degree on "Exploring capacity for youth participation in online public consultation".

===What is the NIYF===
The [http://www.niyf.org/ Northern Ireland Youth Forum (NIYF)] is a key agency in current youth participation activity, developing mechanisms for enabling youth participation across [http://www.discoverireland.com Northern Ireland]. Working with the 14–25 age group, [http://www.niyf.org/ NIYF] currently delivers a number of programmes in various settings, such as youth clubs.

===Doctoral Research===
Smyth 's research explored the capacity for e-enabling appropriate aspects of youth participation in public consultation within existing structures in Northern Ireland. Various stakeholder perspectives were interpreted with a view to developing an online resource offering experiential learning opportunities on citizenship and related themes.

The purpose of the research is to inform the design of appropriate online mechanisms that may be used by [http://www.niyf.org/ NIYF] to complement their existing consultation work. To date, a website using wiki technology has been set up to act as a platform for complementary e-consultation activity and a number of online technology interventions have been introduced

To date, a number of issues have emerged: Recruitment and maintaining interest, and Child Protection

== [[NIYF: Respondent Recruitment]] ==
Recruiting respondents for a consultation process and maintaining interest throughout the process is a key issue. Participants need to know why it is important for them to take part. What specifically is at stake? In what way will their contributon, and its impact, count?

== [[NIYF & Child Protection]] ==
The nature of the [http://www.niyf.org/ NIYF] work with young people carries with it legal and ethical responsibilities in terms of providing a safe and secure environment free from the threat of harm. Upon introducing these online methods it became quickly apparent that a careful approach would be necessary in order to ensure that these requirements were met.

Despite a desire on the part of Smyth and [http://www.niyf.org/ NIYF] to roll out the online consultation activity to as wide an audience as possible, child protection considerations have hindered this development.

== [[NIYF Consultation: Perceived Benefits]] ==
A number of perceived benefits have been observed to date:
*Young participant expressed enthusiasm for the discussion forum environment, and,
*The Consultation coordinator commented on the usefulness of having such a platform.

Youth e-consultations at NIYF

2008-04-14T21:42:25Z

Jjh:

== [[NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation]] ==

This study was undertaken by ECRG Project sponsorsed Michele Smyth, reading for a PhD degree on "Exploring capacity for youth participation in online public consultation".

===What is the NIYF===
The [http://www.niyf.org/ Northern Ireland Youth Forum (NIYF)] is a key agency in current youth participation activity, developing mechanisms for enabling youth participation across [http://www.discoverireland.com Northern Ireland]. Working with the 14–25 age group, [http://www.niyf.org/ NIYF] currently delivers a number of programmes in various settings, such as youth clubs.

===Doctoral Research===
Smyth 's research explored the capacity for e-enabling appropriate aspects of youth participation in public consultation within existing structures in Northern Ireland. Various stakeholder perspectives were interpreted with a view to developing an online resource offering experiential learning opportunities on citizenship and related themes.

The purpose of the research is to inform the design of appropriate online mechanisms that may be used by [http://www.niyf.org/ NIYF] to complement their existing consultation work. To date, a website using wiki technology has been set up to act as a platform for complementary e-consultation activity and a number of online technology interventions have been introduced

To date, a number of issues have emerged: Recruitment and maintaining interest, and Child Protection

== [[NIYF: Respondent Recruitment]] ==
Recruiting respondents for a consultation process and maintaining interest throughout the process is a key issue. Participants need to know why it is important for them to take part. What specifically is at stake? In what way will their contributon, and its impact, count?

== [[NIYF & Child Protection]] ==
The nature of the [http://www.niyf.org/ NIYF] work with young people carries with it legal and ethical responsibilities in terms of providing a safe and secure environment free from the threat of harm. Upon introducing these online methods it became quickly apparent that a careful approach would be necessary in order to ensure that these requirements were met.

Despite a desire on the part of Smyth and NIYF to roll out the online consultation activity to as wide an audience as possible, child protection considerations have hindered this development.

== [[NIYF Consultation: Perceived Benefits]] ==
A number of perceived benefits have been observed to date:
*Young participant expressed enthusiasm for the discussion forum environment, and,
*The Consultation coordinator commented on the usefulness of having such a platform.

Youth e-consultations at NIYF

2008-04-14T21:39:44Z

Jjh:

== [[NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation]] ==

This study was undertaken by ECRG Project sponsorsed Michele Smyth, reading for a PhD degree on "Exploring capacity for youth participation in online public consultation".

===What is the NIYF===
The [http://www.niyf.org/ Northern Ireland Youth Forum (NIYF)] is a key agency in current youth participation activity, developing mechanisms for enabling youth participation across Northern Ireland. Working with the 14–25 age group, NIYF currently delivers a number of programmes in various settings, such as youth clubs.

Smyth 's research explored the capacity for e-enabling appropriate aspects of youth participation in public consultation within existing structures in Northern Ireland. Various stakeholder perspectives were interpreted with a view to developing an online resource offering experiential learning opportunities on citizenship and related themes.

The purpose of the research is to inform the design of appropriate online mechanisms that may be used by NIYF to complement their existing consultation work. To date, a website using wiki technology has been set up to act as a platform for complementary e-consultation activity and a number of online technology interventions have been introduced

To date, a number of issues have emerged: Recruitment and maintaining interest, and Child Protection

== [[NIYF: Respondent Recruitment]] ==
Recruiting respondents for a consultation process and maintaining interest throughout the process is a key issue. Participants need to know why it is important for them to take part. What specifically is at stake? In what way will their contributon, and its impact, count?

== [[NIYF & Child Protection]] ==
The nature of the NIYF work with young people carries with it legal and ethical responsibilities in terms of providing a safe and secure enviroment free from the threat of harm. Upon introducing these online methods it became quickly apparent that a careful approach would be necessary in order to ensure that these requirements were met.

In particular, the discussion forum software posed a threat because, in its generic form, it allowed anyone to register and post messages. The registration process was, therefore, customised

Another generic aspect of the discussion forum also posed a threat; the fact that the forum was available 24-7 meant that it required particularly vigilant moderation. A facility was set up to alert the moderator via e-mail when a post to the forum had been made.

Despite a desire on the part of Smyth and NIYF to roll out the online consultation activity to as wide an audience as possible, child protection considerations have hindered this development.

== [[NIYF Consultation: Perceived Benefits]] ==
A number of perceived benefits have been observed to date:
*Young participant expressed enthusiasm for the discussion forum environment, and,
*The Consultation coordinator commented on the usefulness of having such a platform.

NIYF Consultation: Perceived Benefits

2008-04-14T21:29:49Z

Jjh: first go

A number of perceived benefits have been observed to date; Young people participating have expressed their enthusiasm for the discussion forum environment. Likewise, the Consultation coordinator commented on the usefulness of having a platform that allowed young people to continue their conversations over a longer period.

NIYF & Child Protection

2008-04-14T21:29:23Z

Jjh: first go

The nature of the NIYF work with young people carries with it legal and ethical responsibilities in terms of providing a safe and secure enviroment free from the threat of harm. Upon introducing these online methods it became quickly apparent that a careful approach would be necessary in order to ensure that these requirements were met. During observations of the face-to-face consultation processes, it became apparent that online mediums' capactiy to facilitate anonymity could prove beneficial, particularly when discussing sensitive issues.

In particular, the discussion forum software posed a threat because, in its generic form, it allowed anyone to register and post messages. It was paramount that those registered must be young people and that they adhere to a code of conduct. The registration process was, therefore, customised to ensure that only young people could register and that they agreed to the code of conduct established. While this process caused a slight delay between registration and participation, the safety of participants was considered by far the more important issue.

Another generic aspect of the discussion forum also posed a threat; the fact that the forum was available 24-7 meant that it required particularly vigilant moderation. A facility was set up to alert the moderator via e-mail when a post to the forum had been made and it became necessary for the moderator to check for posts at regular intervals, including evenings and weekends, in case any inappropriate material was posted.

Despite a desire on the part of Michele and NIYF to roll out the online consultation activity to as wide an audience as possible - child protection considerations have hindered this development. It is now believed that adequate safeguards have been developed incorporating verifiable registration and vigilant moderation with any generic loopholes allowing access having been closed.

NIYF: Respondent Recruitment

2008-04-14T21:29:04Z

Jjh: first go

Recruiting respondents for a consultation process and maintaining interest throughout the process is a key issue. Participants need to know why it is important for them to take part. What specifically is at stake? In what way will their contributon, and its impact, count?

As young people give up some of their free time, NIYF offers incentives to participate in their face-to-face activity when resources permit. Although no incentives for participating online have been offered to date, the possibilities for offering music downloads or other such gratuities may be explored.

NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation

2008-04-14T21:28:39Z

Jjh:

Throughout the course of the ECRG Project, sponsorship has been provided for Michele Smyth to undertake research contributing to the award of a PhD degree in the e-consultation field. The topic for this research is "Exploring capacity for youth participation in online public consultation".

The Northern Ireland Youth Forum (NIYF) is a key agency in current youth participation activity. For over 25 years the forum has been a central force in developing mechanisms for enabling youth participation across Northern Ireland. Working with the 14–25 age group, NIYF currently delivers a number of programmes that characteristically incorporate innovative techniques and a high level of interactivity. These programmes are delivered in various ‘real world’ settings such as youth clubs.

Through discussions with NIYF, a shared research interest in e-enabled approaches to consultation and their potential capacity to enrich current youth consultation activities has been established and access has been negotiated to conduct a research programme. This research explores the capacity for e-enabling appropriate aspects of youth participation in public consultation within existing structures in Northern Ireland. Various stakeholder perspectives will be interpreted with a view to developing an online resource that has the potential to offer experiential learning opportunities on citizenship and its related themes within classroom settings.

Adopting an action research approach, this has involved shadowing a youth worker directly involved in engaging young people in public consultation processes. In keeping with the action research approach, cycles of research activity have been undertaken that observe practice, and then identify and implement appropriate e-enabled interventions. These interventions and emerging issues are observed and these subsequently inform the next cycle in an iterative process.

The purpose of the cyclical enquiry is to iteratively inform the design of appropriate online mechanisms that may be used by NIYF to complement their existing consultation work. During the consultation processes, where appropriate, an intervention in the form of an e-enabled component has been introduced. To date, a website using wiki technology has been set up to act as a platform for complementary e-consultation activity The youth e-consultation website Over the last 12 months a number of online technology interventions have been introduced: discussion forum software and online survey software. The table below serves as a guide for consulting bodies approaching NIYF to undertake a consultation on their behalf, illustrating the opportunities and constraints of the methods listed.

[[Image:NIYF_tech_table.JPG]]

The table illustrates e-consultation methods NIYF currently offers; note, however, that we are continually developing methods and trying out different technologies. Any of these methods can be tailored and combined to fit specific consultation requirements.

OTHER CONSIDERATIONS:

PROMOTION: Whatever the method - it is essential that it be appropriately promoted.

TIMEFRAMES: As a general rule of thumb, an ‘activity’ period of 6-8 weeks should be accommodated within the planned timeframe. Please note this does not include recruitment, setting up & reporting after the event.

INCENTIVES: You may wish to provide incentives for participating, depending on budgetary constraints, as these can encourage participation

NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation

2008-04-14T21:28:09Z

Jjh:

Throughout the course of the ECRG Project, sponsorship has been provided for Michele Smyth to undertake research contributing to the award of a PhD degree in the e-consultation field. The topic for this research is "Exploring capacity for youth participation in online public consultation".

The Northern Ireland Youth Forum (NIYF) is a key agency in current youth participation activity. For over 25 years the forum has been a central force in developing mechanisms for enabling youth participation across Northern Ireland. Working with the 14–25 age group, NIYF currently delivers a number of programmes that characteristically incorporate innovative techniques and a high level of interactivity. These programmes are delivered in various ‘real world’ settings such as youth clubs.

Through discussions with NIYF, a shared research interest in e-enabled approaches to consultation and their potential capacity to enrich current youth consultation activities has been established and access has been negotiated to conduct a research programme. This research explores the capacity for e-enabling appropriate aspects of youth participation in public consultation within existing structures in Northern Ireland. Various stakeholder perspectives will be interpreted with a view to developing an online resource that has the potential to offer experiential learning opportunities on citizenship and its related themes within classroom settings.

Adopting an action research approach, this has involved shadowing a youth worker directly involved in engaging young people in public consultation processes. In keeping with the action research approach, cycles of research activity have been undertaken that observe practice, and then identify and implement appropriate e-enabled interventions. These interventions and emerging issues are observed and these subsequently inform the next cycle in an iterative process.

The purpose of the cyclical enquiry is to iteratively inform the design of appropriate online mechanisms that may be used by NIYF to complement their existing consultation work. During the consultation processes, where appropriate, an intervention in the form of an e-enabled component has been introduced. To date, a website using wiki technology has been set up to act as a platform for complementary e-consultation activity The youth e-consultation website Over the last 12 months a number of online technology interventions have been introduced: discussion forum software and online survey software. The table below serves as a guide for consulting bodies approaching NIYF to undertake a consultation on their behalf, illustrating the opportunities and constraints of the methods listed.

The table illustrates e-consultation methods NIYF currently offers; note, however, that we are continually developing methods and trying out different technologies. Any of these methods can be tailored and combined to fit specific consultation requirements.

OTHER CONSIDERATIONS:

PROMOTION: Whatever the method - it is essential that it be appropriately promoted.

TIMEFRAMES: As a general rule of thumb, an ‘activity’ period of 6-8 weeks should be accommodated within the planned timeframe. Please note this does not include recruitment, setting up & reporting after the event.

INCENTIVES: You may wish to provide incentives for participating, depending on budgetary constraints, as these can encourage participation

NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation

2008-04-14T21:27:45Z

Jjh: first go

Youth e-consultations at NIYF

2008-04-14T21:27:24Z

Jjh:

== [[NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation]] ==

Throughout the course of the ECRG Project, sponsorship has been provided for Michele Smyth to undertake research contributing to the award of a PhD degree in the e-consultation field. The topic for this research is "Exploring capacity for youth participation in online public consultation".

The Northern Ireland Youth Forum (NIYF) is a key agency in current youth participation activity. For over 25 years the forum has been a central force in developing mechanisms for enabling youth participation across Northern Ireland. Working with the 14–25 age group, NIYF currently delivers a number of programmes that characteristically incorporate innovative techniques and a high level of interactivity. These programmes are delivered in various ‘real world’ settings such as youth clubs.

Through discussions with NIYF, a shared research interest in e-enabled approaches to consultation and their potential capacity to enrich current youth consultation activities has been established and access has been negotiated to conduct a research programme. This research explores the capacity for e-enabling appropriate aspects of youth participation in public consultation within existing structures in Northern Ireland. Various stakeholder perspectives will be interpreted with a view to developing an online resource that has the potential to offer experiential learning opportunities on citizenship and its related themes within classroom settings.

Adopting an action research approach, this has involved shadowing a youth worker directly involved in engaging young people in public consultation processes. In keeping with the action research approach, cycles of research activity have been undertaken that observe practice, and then identify and implement appropriate e-enabled interventions. These interventions and emerging issues are observed and these subsequently inform the next cycle in an iterative process.

The purpose of the cyclical enquiry is to iteratively inform the design of appropriate online mechanisms that may be used by NIYF to complement their existing consultation work. During the consultation processes, where appropriate, an intervention in the form of an e-enabled component has been introduced. To date, a website using wiki technology has been set up to act as a platform for complementary e-consultation activity [http://youth.e-consultation.org The youth e-consultation website ]Over the last 12 months a number of online technology interventions have been introduced: discussion forum software and online survey software. The table below serves as a guide for consulting bodies approaching NIYF to undertake a consultation on their behalf, illustrating the opportunities and constraints of the methods listed.

----
[[Image:NIYF_tech_table.JPG]]

The table illustrates e-consultation methods NIYF currently offers; note, however, that we are continually developing methods and trying out different technologies. Any of these methods can be tailored and combined to fit specific consultation requirements.

OTHER CONSIDERATIONS:

PROMOTION: Whatever the method - it is essential that it be appropriately promoted.

TIMEFRAMES: As a general rule of thumb, an ‘activity’ period of 6-8 weeks should be accommodated within the planned timeframe. Please note this does not include recruitment, setting up & reporting after the event.

INCENTIVES: You may wish to provide incentives for participating, depending on budgetary constraints, as these can encourage participation

----

Thus far, a number of issues have emerged:

== [[NIYF: Respondent Recruitment]] ==
Recruiting respondents for a consultation process and maintaining interest throughout the process is a key issue. Participants need to know why it is important for them to take part. What specifically is at stake? In what way will their contributon, and its impact, count?

As young people give up some of their free time, NIYF offers incentives to participate in their face-to-face activity when resources permit. Although no incentives for participating online have been offered to date, the possibilities for offering music downloads or other such gratuities may be explored.

== [[NIYF & Child Protection]] ==
The nature of the NIYF work with young people carries with it legal and ethical responsibilities in terms of providing a safe and secure enviroment free from the threat of harm. Upon introducing these online methods it became quickly apparent that a careful approach would be necessary in order to ensure that these requirements were met. During observations of the face-to-face consultation processes, it became apparent that online mediums' capactiy to facilitate anonymity could prove beneficial, particularly when discussing sensitive issues.

In particular, the discussion forum software posed a threat because, in its generic form, it allowed anyone to register and post messages. It was paramount that those registered must be young people and that they adhere to a code of conduct. The registration process was, therefore, customised to ensure that only young people could register and that they agreed to the code of conduct established. While this process caused a slight delay between registration and participation, the safety of participants was considered by far the more important issue.

Another generic aspect of the discussion forum also posed a threat; the fact that the forum was available 24-7 meant that it required particularly vigilant moderation. A facility was set up to alert the moderator via e-mail when a post to the forum had been made and it became necessary for the moderator to check for posts at regular intervals, including evenings and weekends, in case any inappropriate material was posted.

Despite a desire on the part of Michele and NIYF to roll out the online consultation activity to as wide an audience as possible - child protection considerations have hindered this development. It is now believed that adequate safeguards have been developed incorporating verifiable registration and vigilant moderation with any generic loopholes allowing access having been closed.

== [[NIYF Consultation: Perceived Benefits]] ==
A number of perceived benefits have been observed to date;
Young people participating have expressed their enthusiasm for the discussion forum environment. Likewise, the Consultation coordinator commented on the usefulness of having a platform that allowed young people to continue their conversations over a longer period.

Youth e-consultations at NIYF

2008-04-14T21:26:45Z

Jjh:

== [[NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation]] ==

Throughout the course of the ECRG Project, sponsorship has been provided for Michele Smyth to undertake research contributing to the award of a PhD degree in the e-consultation field. The topic for this research is "Exploring capacity for youth participation in online public consultation".

The Northern Ireland Youth Forum (NIYF) is a key agency in current youth participation activity. For over 25 years the forum has been a central force in developing mechanisms for enabling youth participation across Northern Ireland. Working with the 14–25 age group, NIYF currently delivers a number of programmes that characteristically incorporate innovative techniques and a high level of interactivity. These programmes are delivered in various ‘real world’ settings such as youth clubs.

Through discussions with NIYF, a shared research interest in e-enabled approaches to consultation and their potential capacity to enrich current youth consultation activities has been established and access has been negotiated to conduct a research programme. This research explores the capacity for e-enabling appropriate aspects of youth participation in public consultation within existing structures in Northern Ireland. Various stakeholder perspectives will be interpreted with a view to developing an online resource that has the potential to offer experiential learning opportunities on citizenship and its related themes within classroom settings.

Adopting an action research approach, this has involved shadowing a youth worker directly involved in engaging young people in public consultation processes. In keeping with the action research approach, cycles of research activity have been undertaken that observe practice, and then identify and implement appropriate e-enabled interventions. These interventions and emerging issues are observed and these subsequently inform the next cycle in an iterative process.

The purpose of the cyclical enquiry is to iteratively inform the design of appropriate online mechanisms that may be used by NIYF to complement their existing consultation work. During the consultation processes, where appropriate, an intervention in the form of an e-enabled component has been introduced. To date, a website using wiki technology has been set up to act as a platform for complementary e-consultation activity [http://youth.e-consultation.org The youth e-consultation website ]Over the last 12 months a number of online technology interventions have been introduced: discussion forum software and online survey software. The table below serves as a guide for consulting bodies approaching NIYF to undertake a consultation on their behalf, illustrating the opportunities and constraints of the methods listed.

----
[[Image:NIYF_tech_table.JPG]]

The table illustrates e-consultation methods NIYF currently offers; note, however, that we are continually developing methods and trying out different technologies. Any of these methods can be tailored and combined to fit specific consultation requirements.

OTHER CONSIDERATIONS:

PROMOTION: Whatever the method - it is essential that it be appropriately promoted.

TIMEFRAMES: As a general rule of thumb, an ‘activity’ period of 6-8 weeks should be accommodated within the planned timeframe. Please note this does not include recruitment, setting up & reporting after the event.

INCENTIVES: You may wish to provide incentives for participating, depending on budgetary constraints, as these can encourage participation

----

Thus far, a number of issues have emerged:

== [NIYF: Respondent Recruitment] ==
Recruiting respondents for a consultation process and maintaining interest throughout the process is a key issue. Participants need to know why it is important for them to take part. What specifically is at stake? In what way will their contributon, and its impact, count?

As young people give up some of their free time, NIYF offers incentives to participate in their face-to-face activity when resources permit. Although no incentives for participating online have been offered to date, the possibilities for offering music downloads or other such gratuities may be explored.

== [[NIYF & Child Protection] ==
The nature of the NIYF work with young people carries with it legal and ethical responsibilities in terms of providing a safe and secure enviroment free from the threat of harm. Upon introducing these online methods it became quickly apparent that a careful approach would be necessary in order to ensure that these requirements were met. During observations of the face-to-face consultation processes, it became apparent that online mediums' capactiy to facilitate anonymity could prove beneficial, particularly when discussing sensitive issues.

In particular, the discussion forum software posed a threat because, in its generic form, it allowed anyone to register and post messages. It was paramount that those registered must be young people and that they adhere to a code of conduct. The registration process was, therefore, customised to ensure that only young people could register and that they agreed to the code of conduct established. While this process caused a slight delay between registration and participation, the safety of participants was considered by far the more important issue.

Another generic aspect of the discussion forum also posed a threat; the fact that the forum was available 24-7 meant that it required particularly vigilant moderation. A facility was set up to alert the moderator via e-mail when a post to the forum had been made and it became necessary for the moderator to check for posts at regular intervals, including evenings and weekends, in case any inappropriate material was posted.

Despite a desire on the part of Michele and NIYF to roll out the online consultation activity to as wide an audience as possible - child protection considerations have hindered this development. It is now believed that adequate safeguards have been developed incorporating verifiable registration and vigilant moderation with any generic loopholes allowing access having been closed.

== [NIYF Consultation: Perceived Benefits] ==
A number of perceived benefits have been observed to date;
Young people participating have expressed their enthusiasm for the discussion forum environment. Likewise, the Consultation coordinator commented on the usefulness of having a platform that allowed young people to continue their conversations over a longer period.

Youth e-consultations at NIYF

2008-04-14T21:26:15Z

Jjh: first stab

== [NIYF: Exploring Capacity for E-Enabled Youth Participation in Public Consultation] ==

Throughout the course of the ECRG Project, sponsorship has been provided for Michele Smyth to undertake research contributing to the award of a PhD degree in the e-consultation field. The topic for this research is "Exploring capacity for youth participation in online public consultation".

The Northern Ireland Youth Forum (NIYF) is a key agency in current youth participation activity. For over 25 years the forum has been a central force in developing mechanisms for enabling youth participation across Northern Ireland. Working with the 14–25 age group, NIYF currently delivers a number of programmes that characteristically incorporate innovative techniques and a high level of interactivity. These programmes are delivered in various ‘real world’ settings such as youth clubs.

Through discussions with NIYF, a shared research interest in e-enabled approaches to consultation and their potential capacity to enrich current youth consultation activities has been established and access has been negotiated to conduct a research programme. This research explores the capacity for e-enabling appropriate aspects of youth participation in public consultation within existing structures in Northern Ireland. Various stakeholder perspectives will be interpreted with a view to developing an online resource that has the potential to offer experiential learning opportunities on citizenship and its related themes within classroom settings.

Adopting an action research approach, this has involved shadowing a youth worker directly involved in engaging young people in public consultation processes. In keeping with the action research approach, cycles of research activity have been undertaken that observe practice, and then identify and implement appropriate e-enabled interventions. These interventions and emerging issues are observed and these subsequently inform the next cycle in an iterative process.

The purpose of the cyclical enquiry is to iteratively inform the design of appropriate online mechanisms that may be used by NIYF to complement their existing consultation work. During the consultation processes, where appropriate, an intervention in the form of an e-enabled component has been introduced. To date, a website using wiki technology has been set up to act as a platform for complementary e-consultation activity [http://youth.e-consultation.org The youth e-consultation website ]Over the last 12 months a number of online technology interventions have been introduced: discussion forum software and online survey software. The table below serves as a guide for consulting bodies approaching NIYF to undertake a consultation on their behalf, illustrating the opportunities and constraints of the methods listed.

----
[[Image:NIYF_tech_table.JPG]]

The table illustrates e-consultation methods NIYF currently offers; note, however, that we are continually developing methods and trying out different technologies. Any of these methods can be tailored and combined to fit specific consultation requirements.

OTHER CONSIDERATIONS:

PROMOTION: Whatever the method - it is essential that it be appropriately promoted.

TIMEFRAMES: As a general rule of thumb, an ‘activity’ period of 6-8 weeks should be accommodated within the planned timeframe. Please note this does not include recruitment, setting up & reporting after the event.

INCENTIVES: You may wish to provide incentives for participating, depending on budgetary constraints, as these can encourage participation

----

Thus far, a number of issues have emerged:

== [NIYF: Respondent Recruitment] ==
Recruiting respondents for a consultation process and maintaining interest throughout the process is a key issue. Participants need to know why it is important for them to take part. What specifically is at stake? In what way will their contributon, and its impact, count?

As young people give up some of their free time, NIYF offers incentives to participate in their face-to-face activity when resources permit. Although no incentives for participating online have been offered to date, the possibilities for offering music downloads or other such gratuities may be explored.

== [[NIYF & Child Protection] ==
The nature of the NIYF work with young people carries with it legal and ethical responsibilities in terms of providing a safe and secure enviroment free from the threat of harm. Upon introducing these online methods it became quickly apparent that a careful approach would be necessary in order to ensure that these requirements were met. During observations of the face-to-face consultation processes, it became apparent that online mediums' capactiy to facilitate anonymity could prove beneficial, particularly when discussing sensitive issues.

In particular, the discussion forum software posed a threat because, in its generic form, it allowed anyone to register and post messages. It was paramount that those registered must be young people and that they adhere to a code of conduct. The registration process was, therefore, customised to ensure that only young people could register and that they agreed to the code of conduct established. While this process caused a slight delay between registration and participation, the safety of participants was considered by far the more important issue.

Another generic aspect of the discussion forum also posed a threat; the fact that the forum was available 24-7 meant that it required particularly vigilant moderation. A facility was set up to alert the moderator via e-mail when a post to the forum had been made and it became necessary for the moderator to check for posts at regular intervals, including evenings and weekends, in case any inappropriate material was posted.

Despite a desire on the part of Michele and NIYF to roll out the online consultation activity to as wide an audience as possible - child protection considerations have hindered this development. It is now believed that adequate safeguards have been developed incorporating verifiable registration and vigilant moderation with any generic loopholes allowing access having been closed.

== [NIYF Consultation: Perceived Benefits] ==
A number of perceived benefits have been observed to date;
Young people participating have expressed their enthusiasm for the discussion forum environment. Likewise, the Consultation coordinator commented on the usefulness of having a platform that allowed young people to continue their conversations over a longer period.

Discussion

2008-04-14T21:20:06Z

Jjh:

===Answering a critical question===
'''To what extent is it possible through technology to improve accessibility and usability for those with lower literacy levels?'''

We have shown it is possible to design interfaces that reduce the cognitive burden on consultees, compared to [http://www.pbni.org.uk/annex-a-questionnaire.pdf traditional questionnaires]. This is consistent with a commonly stated principle of human-computer interface design: ''making the interface consistent with the ways the user thinks about the problem''.

Instead of forcing people to translate in their head from questions, to their experience, and back into written answers, let them work with a visualisation that mirrors their mental model. If a consultation is about the location of something, then a map is a natural representation of that.

===Limited application===
Although computer maps have been used in consultations on planning issues for over a decade, few consulting bodies have used them in small consultations, because of the cost of preparing the Geographic Information Systems (GIS). But now that lowcost GIS is easily available, from [http://www.google.com/map/ Google maps] to the open source [http://grass.itc.it/announces/announce_grass623.html GRASS] toolkit, such techniques are becoming more feasible. It only took a couple of days for Ashish Italiya to design a CommunityWalk map for this test, and populate it with data and photographs.

===Some limitations remain with web maps===
Nevertheless, even our map still requires some literacy.

Users need to be able to recognise place names (not too difficult) and then type in short comments (rather harder). For a less literate consultee the interface needs to provide information through more photographs (or drawings) and sound (e.g. click on a location and the site plays a recording of the information and what you have to do). To enter information, there would need to be a choice of pictures to click on, or a way of recording voice and storing it. Such approaches turn the interaction with the consultation into something like a short segment of playing a computer game.

There have been computer games (See [http://www.demgames.org/ demgames]) designed to support e-democracy. The next challenge is interaction design to engage those most excluded by current consultation techniques (both traditional and electronic), but that would involve an entire research programme in itself.

Probation Board of Northern Ireland

2008-01-13T18:37:44Z

Jjh: /* Discussion */

===Overview===
E-consultation is often criticized on the grounds that different groups of people do not have equal access to electronic communications technologies. However, in our modern times, this criticism is a weak one.

E-consultation techniques are complementary to traditional approaches, and seek to improve participation.

In this case example we discuss what can be done to improve accessibility and usability for groups, who may be otherwise excluded from e-consultation.

===[[Probation Board of Northern Ireland Consultation]]===
In 2005, the PBNI were faced with consdierable structural change. They wished to consult widely on the changes to local probation office locations, and reporting centres, across Northern Ireland.

The PBNI sought opinions from several partner groups, political parties, councillors, community groups, and Individual offenders.

The PBNI drew learning from previous consultations. The most signficant challenge was engaging with offenders who had a low literacy, numeracy and other learning difficulties. However, it was recognised that, regardless, such stakeholders were very capable of texting using their mobile phones.

The project team suggested a number of technologies that could be used in their consultations with organisations.

PBNI consultation managers then spent some time over the autumn of 2005 on designing and planning the consultation, with the help of the [http://www.consultationinstitute.org/ Consultation Institute (represented by Stratagem in Ireland)].

The Consultation Institute pointed out the disadvantages of consulting on only one out of nine options (i.e. ex-offenders). In addition, the PBNI needed approval from the its corporate managers, board, and the Northern Ireland Office, which created a long delay.

In the end, the PBNI ran a conventional consultation, without any e-consultation component, between 10 March and 2 June 2006. Rather than introducing a new technology for consulting with offenders during this major consultation, they agreed to work with us to do a usability test of an e-consultation technology with ex-offenders.

===[[Designing an e-consultation interface for ex-offenders]]===
Several technologies were considered usable by people with reading and writing difficulties.
These used voice, mobile text, or graphical interfaces over the Internet. We chose a high-graphics, low text interface to explore the extent of use by ex-offenders. Also, since the PBNI consultation was about the location of probation offices, we chose to use a geographical, map-based interface.

[http://maps.google.co.uk Google Maps' (maps.google.co.uk)] Applicaiton Programme Interface was used, which lets website developers to use their maps in their own applications.

===[[Designing a usability test]]===
This stage involved designing a usability test of an e-consultation interface for ex-offenders. The map-based interface was compared with the conventional paper
questionnaire, [http://www.pbni.org.uk/annex-a-questionnaire.pdf viewable online], produced by the [http://www.pbni.org.uk/ PBNI].

In partnership with the [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)], we recruited a selection of ex-offenders to test the system on 13 April 2006, during a regular IT class within prison.

We designed a sequence of tasks that started with simple familiarization with the map-based
site, then got progressively more involved as the testers gained confidence, until they were
entering comments on the consultation topic.

We compared this with how much of the [http://www.pbni.org.uk/ PBNI] questionnaire could be filled in within 15 minutes.

We set up [http://www.techsmith.com/camtasia.asp?CMP=KgoogleCStmhome Camtasia Studio], a computer activity recorder, on one PC. This software produced a video of the screen, including the position of the mouse at any time, and what the tester saw.

The tester spoke aloud during the test, explaining what the user was doing, and noting any difficulties in using the interface. In addition, any comments entered via the keyboard were automatically stored on the server.

After completing the test tasks we asked the testers what they thought of the interface, using the post-test questionnaire, and in a focus group, to discuss their experiences.

===[[Running the usability test]]===
A structured programme guided each e-consultation session. However, there was a very low turnout to scheduled sessions.

A possible explanation is that non-offenders do not know anything about probation offices, and probationers are less willing to help the PBNI.

We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening.

For those who participated, only limited assistance was requried. For example, help entering the comments on the map or reading the [http://www.pbni.org.uk/annex-a-questionnaire.pdf consultation questionnaire]. Otherwise, there were few difficulties encountered.

===[[Usability test results]]===
Using the test protocol, we compared the innovative map-based e-consultation technique with the [http://www.pbni.org.uk/annex-a-questionnaire.pdf conventional paper questionnaire].

The web map was better than the [http://www.pbni.org.uk/annex-a-questionnaire.pdf paper questionnaire] on all but one criterion. The exception was the organization of information on the page.

During the tests, there were frequent complaints about the [http://www.pbni.org.uk/annex-a-questionnaire.pdf paper questionnaire]. Q5 was particularly hard to grasp. It reads like a university or A-level examination question. However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer.

In contrast, there were few problems when using the on-line map. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.

===[[Discussion]]===
This case has shown it is possible to design interfaces that reduce the cognitive burden on consultees, compared to traditional questionnaires. This is consistent with a commonly stated principle of human-computer interface design: ''making the interface consistent with the ways the user thinks about the problem''.

Nevertheless, even our map still requires some literacy. sers need to be able to recognise place names and then type in short comments. For a less literate consultee the interface needs to provide information through more photographs and sound.

Although computer maps have been used in consultations on planning issues for over a decade, few consulting bodies have used them in small consultations, because of the cost of preparing the Geographic Information Systems (GIS). But now that lowcost GIS is easily available, from Google maps to the open source GRASS toolkit, such techniques are becoming more feasible. There have even been computer games (See demgames) designed to support e-democracy.

Discussion

2008-01-13T18:34:44Z

Jjh: /* Some limiations remain with web maps */

===Answering a critical question===
'''To what extent is it possible through technology to improve accessibility and usability for those with lower literacy levels?'''

We have shown it is possible to design interfaces that reduce the cognitive burden on consultees, compared to [http://www.pbni.org.uk/annex-a-questionnaire.pdf traditional questionnaires]. This is consistent with a commonly stated principle of human-computer interface design: ''making the interface consistent with the ways the user thinks about the problem''.

Instead of forcing people to translate in their head from questions, to their experience, and back into written answers, let them work with a visualisation that mirrors their mental model. If a consultation is about the location of something, then a map is a natural representation of that.

===Liimited application===
Although computer maps have been used in consultations on planning issues for over a decade, few consulting bodies have used them in small consultations, because of the cost of preparing the Geographic Information Systems (GIS). But now that lowcost GIS is easily available, from [http://www.google.com/map/ Google maps] to the open source [http://grass.itc.it/announces/announce_grass623.html GRASS] toolkit, such techniques are becoming more feasible. It only took a couple of days for Ashish Italiya to design a CommunityWalk map for this test, and populate it with data and photographs.

===Some limiations remain with web maps===
Nevertheless, even our map still requires some literacy.

Users need to be able to recognise place names (not too difficult) and then type in short comments (rather harder). For a less literate consultee the interface needs to provide information through more photographs (or drawings) and sound (e.g. click on a location and the site plays a recording of the information and what you have to do). To enter information, there would need to be a choice of pictures to click on, or a way of recording voice and storing it. Such approaches turn the interaction with the consultation into something like a short segment of playing a computer game.

There have been computer games (See [http://www.demgames.org/ demgames]) designed to support e-democracy. The next challenge is interaction design to engage those most excluded by current consultation techniques (both traditional and electronic), but that would involve an entire research programme in itself.

Discussion

2008-01-13T18:34:22Z

Jjh:

===Answering a critical question===
'''To what extent is it possible through technology to improve accessibility and usability for those with lower literacy levels?'''

We have shown it is possible to design interfaces that reduce the cognitive burden on consultees, compared to [http://www.pbni.org.uk/annex-a-questionnaire.pdf traditional questionnaires]. This is consistent with a commonly stated principle of human-computer interface design: ''making the interface consistent with the ways the user thinks about the problem''.

Instead of forcing people to translate in their head from questions, to their experience, and back into written answers, let them work with a visualisation that mirrors their mental model. If a consultation is about the location of something, then a map is a natural representation of that.

===Liimited application===
Although computer maps have been used in consultations on planning issues for over a decade, few consulting bodies have used them in small consultations, because of the cost of preparing the Geographic Information Systems (GIS). But now that lowcost GIS is easily available, from [http://www.google.com/map/ Google maps] to the open source [http://grass.itc.it/announces/announce_grass623.html GRASS] toolkit, such techniques are becoming more feasible. It only took a couple of days for Ashish Italiya to design a CommunityWalk map for this test, and populate it with data and photographs.

===Some limiations remain with web maps===
Nevertheless, even our map still requires some literacy.

Users need to be able to recognise place names (not too difficult) and then type in short comments (rather harder). For a less literate consultee the interface needs to provide information through more photographs (or drawings) and sound (e.g. click on a location and the site plays a recording of the information and what you have to do). To enter information, there would need to be a choice of pictures to click on, or a way of recording voice and storing it. Such approaches turn the interaction with the consultation into something like a short segment of playing a computer game.

There have been computer games (See [http://www.demgames.org/ Dem Games]) designed to support e-democracy. The next challenge is interaction design to engage those most excluded by current consultation techniques (both traditional and electronic), but that would involve an entire research programme in itself.

Probation Board of Northern Ireland

2008-01-13T18:30:34Z

Jjh: /* Usability test results */

Probation Board of Northern Ireland

2008-01-13T18:30:04Z

Jjh: /* Running the usability test */

===Overview===
E-consultation is often criticized on the grounds that different groups of people do not have equal access to electronic communications technologies. However, in our modern times, this criticism is a weak one.

E-consultation techniques are complementary to traditional approaches, and seek to improve participation.

In this case example we discuss what can be done to improve accessibility and usability for groups, who may be otherwise excluded from e-consultation.

===[[Probation Board of Northern Ireland Consultation]]===
In 2005, the PBNI were faced with consdierable structural change. They wished to consult widely on the changes to local probation office locations, and reporting centres, across Northern Ireland.

The PBNI sought opinions from several partner groups, political parties, councillors, community groups, and Individual offenders.

The PBNI drew learning from previous consultations. The most signficant challenge was engaging with offenders who had a low literacy, numeracy and other learning difficulties. However, it was recognised that, regardless, such stakeholders were very capable of texting using their mobile phones.

The project team suggested a number of technologies that could be used in their consultations with organisations.

PBNI consultation managers then spent some time over the autumn of 2005 on designing and planning the consultation, with the help of the [http://www.consultationinstitute.org/ Consultation Institute (represented by Stratagem in Ireland)].

The Consultation Institute pointed out the disadvantages of consulting on only one out of nine options (i.e. ex-offenders). In addition, the PBNI needed approval from the its corporate managers, board, and the Northern Ireland Office, which created a long delay.

In the end, the PBNI ran a conventional consultation, without any e-consultation component, between 10 March and 2 June 2006. Rather than introducing a new technology for consulting with offenders during this major consultation, they agreed to work with us to do a usability test of an e-consultation technology with ex-offenders.

===[[Designing an e-consultation interface for ex-offenders]]===
Several technologies were considered usable by people with reading and writing difficulties.
These used voice, mobile text, or graphical interfaces over the Internet. We chose a high-graphics, low text interface to explore the extent of use by ex-offenders. Also, since the PBNI consultation was about the location of probation offices, we chose to use a geographical, map-based interface.

[http://maps.google.co.uk Google Maps' (maps.google.co.uk)] Applicaiton Programme Interface was used, which lets website developers to use their maps in their own applications.

===[[Designing a usability test]]===
This stage involved designing a usability test of an e-consultation interface for ex-offenders. The map-based interface was compared with the conventional paper
questionnaire, [http://www.pbni.org.uk/annex-a-questionnaire.pdf viewable online], produced by the [http://www.pbni.org.uk/ PBNI].

In partnership with the [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)], we recruited a selection of ex-offenders to test the system on 13 April 2006, during a regular IT class within prison.

We designed a sequence of tasks that started with simple familiarization with the map-based
site, then got progressively more involved as the testers gained confidence, until they were
entering comments on the consultation topic.

We compared this with how much of the [http://www.pbni.org.uk/ PBNI] questionnaire could be filled in within 15 minutes.

We set up [http://www.techsmith.com/camtasia.asp?CMP=KgoogleCStmhome Camtasia Studio], a computer activity recorder, on one PC. This software produced a video of the screen, including the position of the mouse at any time, and what the tester saw.

The tester spoke aloud during the test, explaining what the user was doing, and noting any difficulties in using the interface. In addition, any comments entered via the keyboard were automatically stored on the server.

After completing the test tasks we asked the testers what they thought of the interface, using the post-test questionnaire, and in a focus group, to discuss their experiences.

===[[Running the usability test]]===
A structured programme guided each e-consultation session. However, there was a very low turnout to scheduled sessions.

A possible explanation is that non-offenders do not know anything about probation offices, and probationers are less willing to help the PBNI.

We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening.

For those who participated, only limited assistance was requried. For example, help entering the comments on the map or reading the [http://www.pbni.org.uk/annex-a-questionnaire.pdf consultation questionnaire]. Otherwise, there were few difficulties encountered.

===[[Usability test results]]===
Using the test protocol, we compared the innovative map-based e-consultation technique with the conventional paper questionnaire.

The web map was better than the paper questionnaire on all but one criterion. The exception was the organization of information on the page.

During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question. However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer.

In contrast, there were few problems when using the on-line map. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.

===[[Discussion]]===

Running the usability test

2008-01-13T18:29:36Z

Jjh: /* User performance */

===Session programme===
For each session we ran through the steps listed in the test protocol. In brief these were:
# Welcome and introduction.
# Pre-test questionnaire (on skills)
# Test the map-based computer interface.
# Post-test questionnaire (on 3)
# Spend up to 15 min. filling in the conventional paper questionnaire as send out by [http://www.niacro.co.uk/ PBNI].
# Post-test questionnaire (on 5)
# Group discussion on usability of 3 and 5.

===Low participation===
On 13 April we set up the software on a number of machines at [http://www.niacro.co.uk/ NIACRO].

As this was the Thursday before Easter Friday, many trainees did not turn up for the session. We were left with one ex-offender in the morning, and one ex-offender and one non-offender in the afternoon. As they were at different stages of the [http://www.niacro.co.uk/ NIACRO] training, their familiarity with computers and the Internet differed.

As only three people turned up before Easter, we left copies of the forms and instructions at
[http://www.niacro.co.uk/ NIACRO] , so that other trainees could be offered the chance to test the software after Easter. But not one trainee did so.

A possible explanation is that non-offenders do not know anything about probation offices, and probationers are less willing to help the [http://www.pbni.org.uk/ PBNI] (or do not trust that their comments will be anonymous). We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening.

===User performance===
We had to help the ex-offenders at times. For example, one needed help entering the comments on the map. Otherwise, there were few difficulties with the map. All needed help in reading the [http://www.pbni.org.uk/annex-a-questionnaire.pdf consultation questionnaire] (at times it used phrases that were more complicated than the language used in tabloid newspapers), and two had difficulties filling it in (one only ticked the boxes, the other had to get a research assistant to write down the free text answers).

Designing a usability test

2008-01-13T18:28:52Z

Jjh:

We set out to compare the usability of this map-based interface with the conventional paper
questionnaire, [http://www.pbni.org.uk/annex-a-questionnaire.pdf viewable online], produced by the [http://www.pbni.org.uk/ PBNI].

==Selecting the testers==
There is no point getting students to test the interface since what we wanted to know was whether ex-offenders, some with low literacy levels, found the map-based interface more or less usable than [http://www.pbni.org.uk/annex-a-questionnaire.pdf current consultation techniques].

====NIACRO====
We needed to recruit ex-offenders to test the system. The [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)] runs IT courses for ex-offenders and others. They have a small computer room in which trainees can develop IT skills that may increase their employability. A number of probationers take these courses. So [http://www.niacro.co.uk/ NIACRO] arranged for some of their trainees to test the interface when they came in for their regular session on 13 April 2006.

==Selecting the tasks==
We designed a sequence of tasks that started with simple familiarization with the map-based
site, then got progressively more involved as the testers gained confidence, until they were
entering comments on the consultation topic.

In brief, the sequence of tasks were:
* Explore the [http://www.pbni.org.uk/ PBNI] [http://www.pbni.org.uk/index/publications/consultation.htm e-consultation website].
* Find your probation office or reporting centre on the map.
* Find the all offices or reporting centres you have been to, and add comments on each place.
* Find your new office and reporting centre, and comment on how it would affect you.

The control task was to complete as much of the [http://www.pbni.org.uk/ PBNI] questionnaire as they could in 15 minutes.

==Collecting test data==
Before the tests started, we asked testers to complete a questionnaire on their familiarity with
computers, the Internet and consultations.

We set up [http://www.techsmith.com/camtasia.asp?CMP=KgoogleCStmhome Camtasia Studio], a computer activity recorder, on one PC. This software produced a video of the screen, including the position of the mouse at any time, and what the tester saw.

The tester spoke aloud during the test, explaining what the user was doing, and noting any difficulties in using the interface. This was recorded on the same Camtasia video, using a microphone plugged into the computer. In addition, the comments entered were automatically stored on the server, so at the end of the session we copied the comments and deleted them from the server.

After completing the test tasks we asked the testers what they thought of the interface, using the post-test questionnaire (See Figure ). We used a similar questionnaire to collect their assessment of the control task, completing a paper questionnaire.

[[Image:PBNI_Postsurvey.png|thumb|Figure: Post-test questionnaire]]

Finally, we invited the testers, in a group, to discuss their experiences in expressing their views through the computer map and the paper questionnaire. We took notes of points raised. The questions asked in the group included:
* ''Now that you have completed all the tasks, please give us your opinions.''
* ''How did you feel about your performance on the tasks overall?''
* ''What would you say was the best thing about this website?''
* ''What would you say the worst thing about this website?''
* ''What do you think of visual design of home page? Navigation? colour?''
* ''Describe your experience of posting your comment on location. Describe what happened.''
* ''What do you think of geographical map for locations in estate review consultation?''
* ''Tell me about what happened when [problem]?''

Probation Board of Northern Ireland

2008-01-13T18:28:10Z

Jjh: /* Designing a usability test */

===Overview===
E-consultation is often criticized on the grounds that different groups of people do not have equal access to electronic communications technologies. However, in our modern times, this criticism is a weak one.

E-consultation techniques are complementary to traditional approaches, and seek to improve participation.

In this case example we discuss what can be done to improve accessibility and usability for groups, who may be otherwise excluded from e-consultation.

===[[Probation Board of Northern Ireland Consultation]]===
In 2005, the PBNI were faced with consdierable structural change. They wished to consult widely on the changes to local probation office locations, and reporting centres, across Northern Ireland.

The PBNI sought opinions from several partner groups, political parties, councillors, community groups, and Individual offenders.

The PBNI drew learning from previous consultations. The most signficant challenge was engaging with offenders who had a low literacy, numeracy and other learning difficulties. However, it was recognised that, regardless, such stakeholders were very capable of texting using their mobile phones.

The project team suggested a number of technologies that could be used in their consultations with organisations.

PBNI consultation managers then spent some time over the autumn of 2005 on designing and planning the consultation, with the help of the [http://www.consultationinstitute.org/ Consultation Institute (represented by Stratagem in Ireland)].

The Consultation Institute pointed out the disadvantages of consulting on only one out of nine options (i.e. ex-offenders). In addition, the PBNI needed approval from the its corporate managers, board, and the Northern Ireland Office, which created a long delay.

In the end, the PBNI ran a conventional consultation, without any e-consultation component, between 10 March and 2 June 2006. Rather than introducing a new technology for consulting with offenders during this major consultation, they agreed to work with us to do a usability test of an e-consultation technology with ex-offenders.

===[[Designing an e-consultation interface for ex-offenders]]===
Several technologies were considered usable by people with reading and writing difficulties.
These used voice, mobile text, or graphical interfaces over the Internet. We chose a high-graphics, low text interface to explore the extent of use by ex-offenders. Also, since the PBNI consultation was about the location of probation offices, we chose to use a geographical, map-based interface.

[http://maps.google.co.uk Google Maps' (maps.google.co.uk)] Applicaiton Programme Interface was used, which lets website developers to use their maps in their own applications.

===[[Designing a usability test]]===
This stage involved designing a usability test of an e-consultation interface for ex-offenders. The map-based interface was compared with the conventional paper
questionnaire, [http://www.pbni.org.uk/annex-a-questionnaire.pdf viewable online], produced by the [http://www.pbni.org.uk/ PBNI].

In partnership with the [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)], we recruited a selection of ex-offenders to test the system on 13 April 2006, during a regular IT class within prison.

We designed a sequence of tasks that started with simple familiarization with the map-based
site, then got progressively more involved as the testers gained confidence, until they were
entering comments on the consultation topic.

We compared this with how much of the [http://www.pbni.org.uk/ PBNI] questionnaire could be filled in within 15 minutes.

We set up [http://www.techsmith.com/camtasia.asp?CMP=KgoogleCStmhome Camtasia Studio], a computer activity recorder, on one PC. This software produced a video of the screen, including the position of the mouse at any time, and what the tester saw.

The tester spoke aloud during the test, explaining what the user was doing, and noting any difficulties in using the interface. In addition, any comments entered via the keyboard were automatically stored on the server.

After completing the test tasks we asked the testers what they thought of the interface, using the post-test questionnaire, and in a focus group, to discuss their experiences.

===[[Running the usability test]]===
A structured programme guided each e-consultation session. However, there was a very low turnout to scheduled sessions.

A possible explanation is that non-offenders do not know anything about probation offices, and probationers are less willing to help the PBNI.

We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening.

For those who participated, only limited assistance was requried. For example, help entering the comments on the map or reading the consultation questionnaire . Otherwise, there were few difficulties encountered.

===[[Usability test results]]===
Using the test protocol, we compared the innovative map-based e-consultation technique with the conventional paper questionnaire.

The web map was better than the paper questionnaire on all but one criterion. The exception was the organization of information on the page.

During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question. However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer.

In contrast, there were few problems when using the on-line map. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.

===[[Discussion]]===

Discussion

2008-01-13T18:26:33Z

Jjh:

===Answering a critical question===
'''To what extent is it possible through technology to improve accessibility and usability for those with lower literacy levels?'''

We have shown it is possible to design interfaces that reduce the cognitive burden on consultees, compared to traditional questionnaires. This is consistent with a commonly stated principle of human-computer interface design: ''making the interface consistent with the ways the user thinks about the problem''.

Instead of forcing people to translate in their head from questions, to their experience, and back into written answers, let them work with a visualisation that mirrors their mental model. If a consultation is about the location of something, then a map is a natural representation of that.

===Liimited application===
Although computer maps have been used in consultations on planning issues for over a decade, few consulting bodies have used them in small consultations, because of the cost of preparing the Geographic Information Systems (GIS). But now that lowcost GIS is easily available, from [http://www.google.com/map/ Google maps] to the open source [http://grass.itc.it/announces/announce_grass623.html GRASS] toolkit, such techniques are becoming more feasible. It only took a couple of days for Ashish Italiya to design a CommunityWalk map for this test, and populate it with data and photographs.

===Some limiations===
Nevertheless, even our map still requires some literacy.

Users need to be able to recognise place names (not too difficult) and then type in short comments (rather harder). For a less literate consultee the interface needs to provide information through more photographs (or drawings) and sound (e.g. click on a location and the site plays a recording of the information and what you have to do). To enter information, there would need to be a choice of pictures to click on, or a way of recording voice and storing it. Such approaches turn the interaction with the consultation into something like a short segment of playing a computer game. There have been computer games designed to support e-democracy. The next challenge is interaction design to engage those most excluded by current consultation techniques (both traditional and electronic), but that would involve an entire research programme in itself.

Probation Board of Northern Ireland

2008-01-13T18:21:56Z

Jjh: /* Usability test results */

===Overview===
E-consultation is often criticized on the grounds that different groups of people do not have equal access to electronic communications technologies. However, in our modern times, this criticism is a weak one.

E-consultation techniques are complementary to traditional approaches, and seek to improve participation.

In this case example we discuss what can be done to improve accessibility and usability for groups, who may be otherwise excluded from e-consultation.

===[[Probation Board of Northern Ireland Consultation]]===
In 2005, the PBNI were faced with consdierable structural change. They wished to consult widely on the changes to local probation office locations, and reporting centres, across Northern Ireland.

The PBNI sought opinions from several partner groups, political parties, councillors, community groups, and Individual offenders.

The PBNI drew learning from previous consultations. The most signficant challenge was engaging with offenders who had a low literacy, numeracy and other learning difficulties. However, it was recognised that, regardless, such stakeholders were very capable of texting using their mobile phones.

The project team suggested a number of technologies that could be used in their consultations with organisations.

PBNI consultation managers then spent some time over the autumn of 2005 on designing and planning the consultation, with the help of the [http://www.consultationinstitute.org/ Consultation Institute (represented by Stratagem in Ireland)].

The Consultation Institute pointed out the disadvantages of consulting on only one out of nine options (i.e. ex-offenders). In addition, the PBNI needed approval from the its corporate managers, board, and the Northern Ireland Office, which created a long delay.

In the end, the PBNI ran a conventional consultation, without any e-consultation component, between 10 March and 2 June 2006. Rather than introducing a new technology for consulting with offenders during this major consultation, they agreed to work with us to do a usability test of an e-consultation technology with ex-offenders.

===[[Designing an e-consultation interface for ex-offenders]]===
Several technologies were considered usable by people with reading and writing difficulties.
These used voice, mobile text, or graphical interfaces over the Internet. We chose a high-graphics, low text interface to explore the extent of use by ex-offenders. Also, since the PBNI consultation was about the location of probation offices, we chose to use a geographical, map-based interface.

[http://maps.google.co.uk Google Maps' (maps.google.co.uk)] Applicaiton Programme Interface was used, which lets website developers to use their maps in their own applications.

===[[Designing a usability test]]===
This stage involved designing a usability test of an e-consultation interface for ex-offenders. The map-based interface was compared with the conventional paper
questionnaire produced by the [http://www.pbni.org.uk/ PBNI].

In partnership with the [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)], we recruited a selection of ex-offenders to test the system on 13 April 2006, during a regular IT class within prison.

We designed a sequence of tasks that started with simple familiarization with the map-based
site, then got progressively more involved as the testers gained confidence, until they were
entering comments on the consultation topic.

We compared this with how much of the [http://www.pbni.org.uk/ PBNI] questionnaire could be filled in within 15 minutes.

We set up [http://www.techsmith.com/camtasia.asp?CMP=KgoogleCStmhome Camtasia Studio], a computer activity recorder, on one PC. This software produced a video of the screen, including the position of the mouse at any time, and what the tester saw.

The tester spoke aloud during the test, explaining what the user was doing, and noting any difficulties in using the interface. In addition, any comments entered via the keyboard were automatically stored on the server.

After completing the test tasks we asked the testers what they thought of the interface, using the post-test questionnaire, and in a focus group, to discuss their experiences.

===[[Running the usability test]]===
A structured programme guided each e-consultation session. However, there was a very low turnout to scheduled sessions.

A possible explanation is that non-offenders do not know anything about probation offices, and probationers are less willing to help the PBNI.

We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening.

For those who participated, only limited assistance was requried. For example, help entering the comments on the map or reading the consultation questionnaire . Otherwise, there were few difficulties encountered.

===[[Usability test results]]===
Using the test protocol, we compared the innovative map-based e-consultation technique with the conventional paper questionnaire.

The web map was better than the paper questionnaire on all but one criterion. The exception was the organization of information on the page.

During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question. However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer.

In contrast, there were few problems when using the on-line map. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.

===[[Discussion]]===

Usability test results

2008-01-13T18:21:00Z

Jjh:

==Test Method==
Using the test protocol, it was possible to compare the innovative map-based e-consultation
technique with the conventional paper questionnaire used in the [http://www.pbni.org.uk/ Probation Board] consultation. The comparison is shown in another chart (See Figure 2)

[[Image:PBNI_Consult_Chart.png|thumb|Figure 1: Chart]]

[[Image:Web_map_test_results.png|thumb|Figure 2: Web map vs. Paper Questionnaires]]

The chart shows the difference between the post-test scores given to the web map and the paper consultation questionnaire. The scores were 5 for strongly agree down to 1 for strongly disagree. The difference can range from +4 to -4. Where the question was a negative one, the difference was taken away from 0, so that in all cases +4 is better for the web map, and -4 is worse.

==Test Results==

Now for two of the testers (R and S) (See Figure 1), the web map was better than the paper questionnaire on all but one criterion. The exception was the organization of information on the page.

For those with less Internet experience, the questionnaire was better. But on every other criterion of ease of use, the map-based interface was better.

The third tester (P) was over 60, and had had less experience of computers and the Internet. On some criteria he found the paper questionnaire easier (based on his experience of filling in forms in the past). Nevertheless, he was more satisfied overall with the new web map than with the traditional questionnaire.

During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question:

''5. PBNI Equality Scheme has given a commitment that in carrying out its functions it will have due regard to the need to promote equality of opportunity between people who fall under the following groups as stated in s75 of the Northern Ireland Act 1998. To help us consider potential inequalities please identify the people in the following groups who you believe will be most affected by the proposed changes (for staff and service users):- Religious belief: Positive [ ] Negative [ ] ..''

However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer.

In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.

The following are some sample comments added to locations on the map.
*''Alderwood House (ISU). There is car parking, and bus and shopping. I feel safe to go to office. There is good services offer in office. Some community perception and people might not go over there. —P''
*''Belfast court (Headquarter). They would be affected by bus strick. They would try to find another way to visit a location—P''
*''lt is just convienet to people living in belfast to go to both courts beside center.—P
*''Ormeau Road There is no disadvantage to this office, staff are very helpful, safe room to speack to probation office.—P''
*''It is parking for people, easy for bues and shops.—P''
*''Shankill Road. The marker wasnt near the location. i live on this road and i got a bit confused finding it on the map.The marker should be on the location.—S''
*''Lisburn Office. found the location i needed. needs no improvements.—S''
*''Newtownards there needs to be more improvement on the map details. if i was really looking for the location i wouldnt get any where because its confusing.—S''

These comments relate directly to the issues of the consultation, focused on specific sites. By starting from the particular, rather than requiring consultees to give general comments, it is both easier for consultees to express their views, and consulters to understand and analyse them.

Designing a usability test

2008-01-13T18:17:01Z

Jjh: /* Collecting test data */

We set out to compare the usability of this map-based interface with the conventional paper
questionnaire produced by the [http://www.pbni.org.uk/ PBNI].

==Selecting the testers==
There is no point getting students to test the interface since what we wanted to know was whether ex-offenders, some with low literacy levels, found the map-based interface more or less usable than current consultation techniques.

====NIACRO====
We needed to recruit ex-offenders to test the system. The [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)] runs IT courses for ex-offenders and others. They have a small computer room in which trainees can develop IT skills that may increase their employability. A number of probationers take these courses. So [http://www.niacro.co.uk/ NIACRO] arranged for some of their trainees to test the interface when they came in for their regular session on 13 April 2006.

==Selecting the tasks==
We designed a sequence of tasks that started with simple familiarization with the map-based
site, then got progressively more involved as the testers gained confidence, until they were
entering comments on the consultation topic.

In brief, the sequence of tasks were:
* Explore the [http://www.pbni.org.uk/ PBNI] [http://www.pbni.org.uk/index/publications/consultation.htm e-consultation website].
* Find your probation office or reporting centre on the map.
* Find the all offices or reporting centres you have been to, and add comments on each place.
* Find your new office and reporting centre, and comment on how it would affect you.

The control task was to complete as much of the [http://www.pbni.org.uk/ PBNI] questionnaire as they could in 15 minutes.

==Collecting test data==
Before the tests started, we asked testers to complete a questionnaire on their familiarity with
computers, the Internet and consultations.

We set up [http://www.techsmith.com/camtasia.asp?CMP=KgoogleCStmhome Camtasia Studio], a computer activity recorder, on one PC. This software produced a video of the screen, including the position of the mouse at any time, and what the tester saw.

The tester spoke aloud during the test, explaining what the user was doing, and noting any difficulties in using the interface. This was recorded on the same Camtasia video, using a microphone plugged into the computer. In addition, the comments entered were automatically stored on the server, so at the end of the session we copied the comments and deleted them from the server.

After completing the test tasks we asked the testers what they thought of the interface, using the post-test questionnaire (See Figure ). We used a similar questionnaire to collect their assessment of the control task, completing a paper questionnaire.

[[Image:PBNI_Postsurvey.png|thumb|Figure: Post-test questionnaire]]

Finally, we invited the testers, in a group, to discuss their experiences in expressing their views through the computer map and the paper questionnaire. We took notes of points raised. The questions asked in the group included:
* ''Now that you have completed all the tasks, please give us your opinions.''
* ''How did you feel about your performance on the tasks overall?''
* ''What would you say was the best thing about this website?''
* ''What would you say the worst thing about this website?''
* ''What do you think of visual design of home page? Navigation? colour?''
* ''Describe your experience of posting your comment on location. Describe what happened.''
* ''What do you think of geographical map for locations in estate review consultation?''
* ''Tell me about what happened when [problem]?''

File:PBNI Postsurvey.png

2008-01-13T18:15:35Z

Jjh:

Designing a usability test

2008-01-13T18:15:24Z

Jjh: /* Collecting test data */

We set out to compare the usability of this map-based interface with the conventional paper
questionnaire produced by the [http://www.pbni.org.uk/ PBNI].

==Selecting the testers==
There is no point getting students to test the interface since what we wanted to know was whether ex-offenders, some with low literacy levels, found the map-based interface more or less usable than current consultation techniques.

====NIACRO====
We needed to recruit ex-offenders to test the system. The [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)] runs IT courses for ex-offenders and others. They have a small computer room in which trainees can develop IT skills that may increase their employability. A number of probationers take these courses. So [http://www.niacro.co.uk/ NIACRO] arranged for some of their trainees to test the interface when they came in for their regular session on 13 April 2006.

==Selecting the tasks==
We designed a sequence of tasks that started with simple familiarization with the map-based
site, then got progressively more involved as the testers gained confidence, until they were
entering comments on the consultation topic.

In brief, the sequence of tasks were:
* Explore the [http://www.pbni.org.uk/ PBNI] [http://www.pbni.org.uk/index/publications/consultation.htm e-consultation website].
* Find your probation office or reporting centre on the map.
* Find the all offices or reporting centres you have been to, and add comments on each place.
* Find your new office and reporting centre, and comment on how it would affect you.

The control task was to complete as much of the [http://www.pbni.org.uk/ PBNI] questionnaire as they could in 15 minutes.

==Collecting test data==
Before the tests started, we asked testers to complete a questionnaire on their familiarity with
computers, the Internet and consultations.

We set up [http://www.techsmith.com/camtasia.asp?CMP=KgoogleCStmhome Camtasia Studio], a computer activity recorder, on one PC. This software produced a video of the screen, including the position of the mouse at any time, and what the tester saw.

The tester spoke aloud during the test, explaining what the user was doing, and noting any difficulties in using the interface. This was recorded on the same Camtasia video, using a microphone plugged into the computer. In addition, the comments entered were automatically stored on the server, so at the end of the session we copied the comments and deleted them from the server.

After completing the test tasks we asked the testers what they thought of the interface, using the post-test questionnaire (See Figure ). We used a similar questionnaire to collect their assessment of the control task, completing a paper questionnaire.

[[Image:|thumb|]]

Finally, we invited the testers, in a group, to discuss their experiences in expressing their views through the computer map and the paper questionnaire. We took notes of points raised. The questions asked in the group included:
* ''Now that you have completed all the tasks, please give us your opinions.''
* ''How did you feel about your performance on the tasks overall?''
* ''What would you say was the best thing about this website?''
* ''What would you say the worst thing about this website?''
* ''What do you think of visual design of home page? Navigation? colour?''
* ''Describe your experience of posting your comment on location. Describe what happened.''
* ''What do you think of geographical map for locations in estate review consultation?''
* ''Tell me about what happened when [problem]?''

Usability test results

2008-01-13T18:12:44Z

Jjh: /* Test Results */

==Test Method==
Using the test protocol, it was possible to compare the innovative map-based e-consultation
technique with the conventional paper questionnaire used in the [http://www.pbni.org.uk/ Probation Board] consultation. The comparison is shown in another chart (See Figure 2)

[[Image:PBNI_Consult_Chart.png|thumb|Figure 1: Chart]]

[[Image:Web_map_test_results.png|thumb|Figure 2: Web map vs. Paper Questionnaires]]

The chart shows the difference between the post-test scores given to the web map and the paper consultation questionnaire. The scores were 5 for strongly agree down to 1 for strongly disagree. The difference can range from +4 to -4. Where the question was a negative one, the difference was taken away from 0, so that in all cases +4 is better for the web map, and -4 is worse.

==Test Results==
Scores suggest that the the paper questionnaire was considered as better.

Now for two of the testers (R and S) (See Figure 1), the web map was better than the paper questionnaire on all
but one criterion. The exception was the organization of information on the page.

For those with less Internet experience, the questionnaire was better. But on every other criterion of ease of use, the map-based interface was better.

The third tester (P) was over 60, and had had less experience of computers and the Internet. On some criteria he found the paper questionnaire easier (based on his experience of filling in forms in the past). Nevertheless, he was more satisfied overall with the new web map than with the traditional questionnaire.

During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question:

''5. PBNI Equality Scheme has given a commitment that in carrying out its functions it will have due regard to the need to promote equality of opportunity between people who fall under the following groups as stated in s75 of the Northern Ireland Act 1998. To help us consider potential inequalities please identify the people in the following groups who you believe will be most affected by the proposed changes (for staff and service users):- Religious belief: Positive [ ] Negative [ ] ..''

However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer.

In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.

The following are some sample comments added to locations on the map.
*''Alderwood House (ISU). There is car parking, and bus and shopping. I feel safe to go to office. There is good services offer in office. Some community perception and people might not go over there. —P''
*''Belfast court (Headquarter). They would be affected by bus strick. They would try to find another way to visit a location—P''
*''lt is just convienet to people living in belfast to go to both courts beside center.—P
*''Ormeau Road There is no disadvantage to this office, staff are very helpful, safe room to speack to probation office.—P''
*''It is parking for people, easy for bues and shops.—P''
*''Shankill Road. The marker wasnt near the location. i live on this road and i got a bit confused finding it on the map.The marker should be on the location.—S''
*''Lisburn Office. found the location i needed. needs no improvements.—S''
*''Newtownards there needs to be more improvement on the map details. if i was really looking for the location i wouldnt get any where because its confusing.—S''

These comments relate directly to the issues of the consultation, focused on specific sites. By starting from the particular, rather than requiring consultees to give general comments, it is both easier for consultees to express their views, and consulters to understand and analyse them.

Usability test results

2008-01-13T18:12:27Z

Jjh:

==Test Method==
Using the test protocol, it was possible to compare the innovative map-based e-consultation
technique with the conventional paper questionnaire used in the [http://www.pbni.org.uk/ Probation Board] consultation. The comparison is shown in another chart (See Figure 2)

[[Image:PBNI_Consult_Chart.png|thumb|Figure 1: Chart]]

[[Image:Web_map_test_results.png|thumb|Figure 2: Web map vs. Paper Questionnaires]]

The chart shows the difference between the post-test scores given to the web map and the paper consultation questionnaire. The scores were 5 for strongly agree down to 1 for strongly disagree. The difference can range from +4 to -4. Where the question was a negative one, the difference was taken away from 0, so that in all cases +4 is better for the web map, and -4 is worse.

==Test Results==
Scores suggest that the the paper questionnaire was considered as better.

Now for two of the testers (R and S) (See Figure 1), the web map was better than the paper questionnaire on all
but one criterion. The exception was the organization of information on the page.

For those with less Internet experience, the questionnaire was better. But on every other criterion of ease of use, the map-based interface was better.

The third tester (P) was over 60, and had had less experience of computers and the Internet. On some criteria he found the paper questionnaire easier (based on his experience of filling in forms in the past). Nevertheless, he was more satisfied overall with the new web map than with the traditional questionnaire.

During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question:

''5. PBNI Equality Scheme has given a commitment that in carrying out its functions it will have due regard to the need to promote equality of opportunity between people who fall under the following groups as stated in s75 of the Northern Ireland Act 1998. To help us consider potential inequalities please identify the people in the following groups who you believe will be most affected by the proposed changes (for staff and service users):- Religious belief: Positive [ ] Negative [ ] ..''

However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer.

In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.

The following are some sample comments added to locations on the map.
*''Alderwood House (ISU). There is car parking, and bus and shopping. I feel safe to go to office. There is good services offer in office. Some community perception and people might not go over there. —P''
*''Belfast court (Headquarter). They would be affected by bus strick. They would try to find another way to visit a location—P''
*''lt is just convienet to people living in belfast to go to both courts beside center.—P
Ormeau Road There is no disadvantage to this office, staff are very helpful, safe room to speack to probation office.—P''
*''It is parking for people, easy for bues and shops.—P''
*''Shankill Road. The marker wasnt near the location. i live on this road and i got a bit confused finding it on the map.The marker should be on the location.—S''
*''Lisburn Office. found the location i needed. needs no improvements.—S''
*''Newtownards there needs to be more improvement on the map details. if i was really looking for the location i wouldnt get any where because its confusing.—S''

These comments relate directly to the issues of the consultation, focused on specific sites. By starting from the particular, rather than requiring consultees to give general comments, it is both easier for consultees to express their views, and consulters to understand and analyse them.

File:Web map test results.png

2008-01-13T18:09:22Z

Jjh:

Usability test results

2008-01-13T18:09:10Z

Jjh:

==Test Method==
Using the test protocol, it was possible to compare the innovative map-based e-consultation
technique with the conventional paper questionnaire used in the [http://www.pbni.org.uk/ Probation Board] consultation. The comparison is shown in another chart (See Figure 2)

[[Image:PBNI_Consult_Chart.png|thumb|Figure 1: Chart]]

[[Image:Web_map_test_results.png|thumb|Figure 2: Web map vs. Paper Questionnaires]]

The chart shows the difference between the post-test scores given to the web map and the paper consultation questionnaire. The scores were 5 for strongly agree down to 1 for strongly disagree. The difference can range from +4 to -4. Where the question was a negative one, the difference was taken away from 0, so that in all cases +4 is better for the web map, and -4 is worse.

==Test Results==
Scores suggest that the the paper questionnaire was considered as better.

Now for two of the testers (R and S) (See Figure 1), the web map was better than the paper questionnaire on all
but one criterion. The exception was the organization of information on the page.

For those with less Internet experience, the questionnaire was better. But on every other criterion of ease of use, the map-based interface was better.

The third tester (P) was over 60, and had had less experience of computers and the Internet. On some criteria he found the paper questionnaire easier (based on his experience of filling in forms in the past). Nevertheless, he was more satisfied overall with the new web map than with the traditional questionnaire.

During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question:

''5. PBNI Equality Scheme has given a commitment that in carrying out its functions it will have due regard to the need to promote equality of opportunity between people who fall under the following groups as stated in s75 of the Northern Ireland Act 1998. To help us consider potential inequalities please identify the people in the following groups who you believe will be most affected by the proposed changes (for staff and service users):- Religious belief: Positive [ ] Negative [ ] ..''

However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer.

In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.

Appendix 7 shows some sample comments added to locations on the map. They relate directly to the issues of the consultation, focused on specific sites. By starting from the particular, rather than requiring consultees to give general comments, it is both easier for consultees to express their views, and consulters to understand and analyse them.

Usability test results

2008-01-13T18:05:27Z

Jjh:

==Test Method==
Using the test protocol, it was possible to compare the innovative map-based e-consultation
technique with the conventional paper questionnaire used in the [http://www.pbni.org.uk/ Probation Board] consultation. The comparison is shown in a chart (See Figure 1)

[[Image:PBNI_Consult_Chart.png|thumb|Figure 1: Chart]]

The chart shows the difference between the post-test scores given to the web map and the paper consultation questionnaire. The scores were 5 for strongly agree down to 1 for strongly disagree. The difference can range from +4 to -4. Where the question was a negative one, the difference was taken away from 0, so that in all cases +4 is better for the web map, and -4 is worse.

==Test Results==
Scores suggest that the the paper questionnaire was considered as better.

Now for two of the testers (R and S), the web map was better than the paper questionnaire on all
but one criterion. The exception was the organization of information on the page.

For those with less Internet experience, the questionnaire was better. But on every other criterion of ease of use, the map-based interface was better.

The third tester (P) was over 60, and had had less experience of computers and the Internet. On some criteria he found the paper questionnaire easier (based on his experience of filling in forms in the past). Nevertheless, he was more satisfied overall with the new web map than with the traditional questionnaire.

During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question:

''5. PBNI Equality Scheme has given a commitment that in carrying out its functions it will have due regard to the need to promote equality of opportunity between people who fall under the following groups as stated in s75 of the Northern Ireland Act 1998. To help us consider potential inequalities please identify the people in the following groups who you believe will be most affected by the proposed changes (for staff and service users):- Religious belief: Positive [ ] Negative [ ] ..''

But even when answering the simpler questions, the testers discussed what the questions mean before
attempting to answer. In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and
move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.
Appendix 7 shows some sample comments added to locations on the map. They relate directly to the issues of the consultation, focused on specific sites. By starting from the particular, rather than requiring consultees to give general comments, it is both easier for consultees to express their views, and consulters to understand and analyse them.

Usability test results

2008-01-13T18:05:10Z

Jjh:

==Test Method==
Using the test protocol, it was possible to compare the innovative map-based e-consultation
technique with the conventional paper questionnaire used in the [http://www.pbni.org.uk/ Probation Board] consultation. The comparison is shown in a chart (See Figure 1)

[[Image:PBNI_Consult_Chart.png|thumb|Figure 1: Chart]]

The chart shows the difference between the post-test scores given to the web map and the paper consultation questionnaire. The scores were 5 for strongly agree down to 1 for strongly disagree. The difference can range from +4 to -4. Where the question was a negative one, the difference was taken away from 0, so that in all cases +4 is better for the web map, and -4 is worse.

==Test Results==
Scores suggest that the the paper questionnaire was considered as better.

Now for two of the testers (R and S), the web map was better than the paper questionnaire on all
but one criterion. The exception was the organization of information on the page.

For those with less Internet experience, the questionnaire was better. But on every other criterion of ease of use, the map-based interface was better.

The third tester (P) was over 60, and had had less experience of computers and the Internet. On some criteria he found the paper questionnaire easier (based on his experience of filling in forms in the past). Nevertheless, he was more satisfied overall with the new web map than with the traditional questionnaire.

During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question:
''5. PBNI Equality Scheme has given a commitment that in carrying out its functions it will have due regard to the need to promote equality of opportunity between people who fall under the following groups as stated in s75 of the Northern Ireland Act 1998. To help us consider potential inequalities please identify the people in the following groups who you believe will be most affected by the proposed changes (for staff and service users):- Religious belief: Positive [ ] Negative [ ] ..''

But even when answering the simpler questions, the testers discussed what the questions mean before
attempting to answer. In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and
move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.
Appendix 7 shows some sample comments added to locations on the map. They relate directly to the issues of the consultation, focused on specific sites. By starting from the particular, rather than requiring consultees to give general comments, it is both easier for consultees to express their views, and consulters to understand and analyse them.

File:PBNI Consult Chart.png

2008-01-13T17:59:38Z

Jjh:

Usability test results

2008-01-13T17:59:30Z

Jjh:

Using this test protocol, it was possible to compare the innovative map-based e-consultation
technique with the conventional paper questionnaire used in the [http://www.pbni.org.uk/ Probation Board] consultation.

[[Image:|thumb|Chart]]

The chart (See Figure 1) shows the difference between the post-test scores given to the web map and the paper consultation questionnaire. The scores were 5 for strongly agree down to 1 for strongly disagree. The difference can range from +4 to -4. Where the question was a negative one, the difference was taken away from 0, so that in all cases +4 is better for the web map, and -4 is worse (i.e. the paper questionnaire is better).

Now for two of the testers (R and S), the web map was better than the paper questionnaire on all
but one criterion.
The exception was the organization of information on the page. For those with less Internet experience, the questionnaire was better. But on every other criterion of ease of use, the map-based interface was better. The third tester (P) was over 60, and had had less experience of computers and the Internet. On some criteria he found the paper questionnaire easier (based on his experience of filling in forms in the past). Nevertheless, he was more satisfied overall with the new web map than with the traditional questionnaire.

During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question.48 But even when answering the simpler questions, the testers discussed what the questions mean before
attempting to answer. In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and
move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.
Appendix 7 shows some sample comments added to locations on the map. They relate directly to the issues of the consultation, focused on specific sites. By starting from the particular, rather than requiring consultees to give general comments, it is both easier for consultees to express their views, and consulters to understand and analyse them.

48 The question started: “5. PBNI Equality Scheme has given a commitment that in carrying out its functions it
will have due regard to the need to promote equality of opportunity between people who fall under the following
groups as stated in s75 of the Northern Ireland Act 1998. To help us consider potential inequalities please
identify the people in the following groups who you believe will be most affected by the proposed changes (for
staff and service users):- Religious belief: Positive [ ] Negative [ ] ...”

Probation Board of Northern Ireland

2008-01-13T17:55:47Z

Jjh: /* Running the usability test */

===Overview===
E-consultation is often criticized on the grounds that different groups of people do not have equal access to electronic communications technologies. However, in our modern times, this criticism is a weak one.

E-consultation techniques are complementary to traditional approaches, and seek to improve participation.

In this case example we discuss what can be done to improve accessibility and usability for groups, who may be otherwise excluded from e-consultation.

===[[Probation Board of Northern Ireland Consultation]]===
In 2005, the PBNI were faced with consdierable structural change. They wished to consult widely on the changes to local probation office locations, and reporting centres, across Northern Ireland.

The PBNI sought opinions from several partner groups, political parties, councillors, community groups, and Individual offenders.

The PBNI drew learning from previous consultations. The most signficant challenge was engaging with offenders who had a low literacy, numeracy and other learning difficulties. However, it was recognised that, regardless, such stakeholders were very capable of texting using their mobile phones.

The project team suggested a number of technologies that could be used in their consultations with organisations.

PBNI consultation managers then spent some time over the autumn of 2005 on designing and planning the consultation, with the help of the [http://www.consultationinstitute.org/ Consultation Institute (represented by Stratagem in Ireland)].

The Consultation Institute pointed out the disadvantages of consulting on only one out of nine options (i.e. ex-offenders). In addition, the PBNI needed approval from the its corporate managers, board, and the Northern Ireland Office, which created a long delay.

In the end, the PBNI ran a conventional consultation, without any e-consultation component, between 10 March and 2 June 2006. Rather than introducing a new technology for consulting with offenders during this major consultation, they agreed to work with us to do a usability test of an e-consultation technology with ex-offenders.

===[[Designing an e-consultation interface for ex-offenders]]===
Several technologies were considered usable by people with reading and writing difficulties.
These used voice, mobile text, or graphical interfaces over the Internet. We chose a high-graphics, low text interface to explore the extent of use by ex-offenders. Also, since the PBNI consultation was about the location of probation offices, we chose to use a geographical, map-based interface.

[http://maps.google.co.uk Google Maps' (maps.google.co.uk)] Applicaiton Programme Interface was used, which lets website developers to use their maps in their own applications.

===[[Designing a usability test]]===
This stage involved designing a usability test of an e-consultation interface for ex-offenders. The map-based interface was compared with the conventional paper
questionnaire produced by the [http://www.pbni.org.uk/ PBNI].

In partnership with the [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)], we recruited a selection of ex-offenders to test the system on 13 April 2006, during a regular IT class within prison.

We designed a sequence of tasks that started with simple familiarization with the map-based
site, then got progressively more involved as the testers gained confidence, until they were
entering comments on the consultation topic.

We compared this with how much of the [http://www.pbni.org.uk/ PBNI] questionnaire could be filled in within 15 minutes.

We set up [http://www.techsmith.com/camtasia.asp?CMP=KgoogleCStmhome Camtasia Studio], a computer activity recorder, on one PC. This software produced a video of the screen, including the position of the mouse at any time, and what the tester saw.

The tester spoke aloud during the test, explaining what the user was doing, and noting any difficulties in using the interface. In addition, any comments entered via the keyboard were automatically stored on the server.

After completing the test tasks we asked the testers what they thought of the interface, using the post-test questionnaire, and in a focus group, to discuss their experiences.

===[[Running the usability test]]===
A structured programme guided each e-consultation session. However, there was a very low turnout to scheduled sessions.

A possible explanation is that non-offenders do not know anything about probation offices, and probationers are less willing to help the PBNI.

We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening.

For those who participated, only limited assistance was requried. For example, help entering the comments on the map or reading the consultation questionnaire . Otherwise, there were few difficulties encountered.

===[[Usability test results]]===

===[[Discussion]]===

Probation Board of Northern Ireland

2008-01-13T17:53:44Z

Jjh:

Probation Board of Northern Ireland

2008-01-13T17:53:14Z

Jjh:

Probation Board of Northern Ireland

2008-01-13T17:52:46Z

Jjh: /* Designing a usability test */

Running the usability test

2008-01-13T17:51:58Z

Jjh: /* Session programme */

===Session programme===
For each session we ran through the steps listed in the test protocol. In brief these were:
# Welcome and introduction.
# Pre-test questionnaire (on skills)
# Test the map-based computer interface.
# Post-test questionnaire (on 3)
# Spend up to 15 min. filling in the conventional paper questionnaire as send out by [http://www.niacro.co.uk/ PBNI].
# Post-test questionnaire (on 5)
# Group discussion on usability of 3 and 5.

===Low participation===
On 13 April we set up the software on a number of machines at [http://www.niacro.co.uk/ NIACRO].

As this was the Thursday before Easter Friday, many trainees did not turn up for the session. We were left with one ex-offender in the morning, and one ex-offender and one non-offender in the afternoon. As they were at different stages of the [http://www.niacro.co.uk/ NIACRO] training, their familiarity with computers and the Internet differed.

As only three people turned up before Easter, we left copies of the forms and instructions at
[http://www.niacro.co.uk/ NIACRO] , so that other trainees could be offered the chance to test the software after Easter. But not one trainee did so.

A possible explanation is that non-offenders do not know anything about probation offices, and probationers are less willing to help the [http://www.pbni.org.uk/ PBNI] (or do not trust that their comments will be anonymous). We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening.

===User performance===
We had to help the ex-offenders at times. For example, one needed help entering the comments on the map. Otherwise, there were few difficulties with the map. All needed help in reading the consultation questionnaire (at times it used phrases that were more complicated than the language used in tabloid newspapers), and two had difficulties filling it in (one only ticked the boxes, the other had to get a research assistant to write down the free text answers).

Running the usability test

2008-01-13T17:51:24Z

Jjh: /* Low participation */

===Session programme===
For each session we ran through the steps listed in the test protocol. In brief these were:
1. Welcome and introduction.
2. Pre-test questionnaire (on skills)
3. Test the map-based computer interface.
4. Post-test questionnaire (on 3)
5. Spend up to 15 min. filling in the conventional paper questionnaire as send out by PBNI.
6. Post-test questionnaire (on 5)
7. Group discussion on usability of 3 and 5.

===Low participation===
On 13 April we set up the software on a number of machines at [http://www.niacro.co.uk/ NIACRO].

As this was the Thursday before Easter Friday, many trainees did not turn up for the session. We were left with one ex-offender in the morning, and one ex-offender and one non-offender in the afternoon. As they were at different stages of the [http://www.niacro.co.uk/ NIACRO] training, their familiarity with computers and the Internet differed.

As only three people turned up before Easter, we left copies of the forms and instructions at
[http://www.niacro.co.uk/ NIACRO] , so that other trainees could be offered the chance to test the software after Easter. But not one trainee did so.

A possible explanation is that non-offenders do not know anything about probation offices, and probationers are less willing to help the [http://www.pbni.org.uk/ PBNI] (or do not trust that their comments will be anonymous). We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening.

===User performance===
We had to help the ex-offenders at times. For example, one needed help entering the comments on the map. Otherwise, there were few difficulties with the map. All needed help in reading the consultation questionnaire (at times it used phrases that were more complicated than the language used in tabloid newspapers), and two had difficulties filling it in (one only ticked the boxes, the other had to get a research assistant to write down the free text answers).

Running the usability test

2008-01-13T17:51:07Z

Jjh:

===Session programme===
For each session we ran through the steps listed in the test protocol. In brief these were:
1. Welcome and introduction.
2. Pre-test questionnaire (on skills)
3. Test the map-based computer interface.
4. Post-test questionnaire (on 3)
5. Spend up to 15 min. filling in the conventional paper questionnaire as send out by PBNI.
6. Post-test questionnaire (on 5)
7. Group discussion on usability of 3 and 5.

===Low participation===
On 13 April we set up the software on a number of machines at NIACRO.

As this was the Thursday before Easter Friday, many trainees did not turn up for the session. We were left with one ex-offender in the morning, and one ex-offender and one non-offender in the afternoon. As they were at different stages of the [http://www.niacro.co.uk/ NIACRO] training, their familiarity with computers and the Internet differed.

As only three people turned up before Easter, we left copies of the forms and instructions at
NIACRO, so that other trainees could be offered the chance to test the software after Easter. But not one trainee did so.

A possible explanation is that non-offenders do not know anything about probation offices, and probationers are less willing to help the [http://www.pbni.org.uk/ PBNI] (or do not trust that their comments will be anonymous). We are looking at ways of following up this preliminary study by designing geographical tasks that young offenders would perceive as more interesting and less threatening.

===User performance===
We had to help the ex-offenders at times. For example, one needed help entering the comments on the map. Otherwise, there were few difficulties with the map. All needed help in reading the consultation questionnaire (at times it used phrases that were more complicated than the language used in tabloid newspapers), and two had difficulties filling it in (one only ticked the boxes, the other had to get a research assistant to write down the free text answers).

Probation Board of Northern Ireland

2008-01-13T17:46:53Z

Jjh: /* Designing a usability test */

Designing a usability test

2008-01-13T17:39:21Z

Jjh: /* NIACRO */

We set out to compare the usability of this map-based interface with the conventional paper
questionnaire produced by the [http://www.pbni.org.uk/ PBNI].

==Selecting the testers==
There is no point getting students to test the interface since what we wanted to know was whether ex-offenders, some with low literacy levels, found the map-based interface more or less usable than current consultation techniques.

====NIACRO====
We needed to recruit ex-offenders to test the system. The [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)] runs IT courses for ex-offenders and others. They have a small computer room in which trainees can develop IT skills that may increase their employability. A number of probationers take these courses. So [http://www.niacro.co.uk/ NIACRO] arranged for some of their trainees to test the interface when they came in for their regular session on 13 April 2006.

==Selecting the tasks==
We designed a sequence of tasks that started with simple familiarization with the map-based
site, then got progressively more involved as the testers gained confidence, until they were
entering comments on the consultation topic.

In brief, the sequence of tasks were:
* Explore the [http://www.pbni.org.uk/ PBNI] [http://www.pbni.org.uk/index/publications/consultation.htm e-consultation website].
* Find your probation office or reporting centre on the map.
* Find the all offices or reporting centres you have been to, and add comments on each place.
* Find your new office and reporting centre, and comment on how it would affect you.

The control task was to complete as much of the [http://www.pbni.org.uk/ PBNI] questionnaire as they could in 15 minutes.

==Collecting test data==
Before the tests started, we asked testers to complete a questionnaire on their familiarity with
computers, the Internet and consultations.

We set up [http://www.techsmith.com/camtasia.asp?CMP=KgoogleCStmhome Camtasia Studio], a computer activity recorder, on one PC. This software produced a video of the screen, including the position of the mouse at any time, and what the tester saw.

The tester spoke aloud during the test, explaining what the user was doing, and noting any difficulties in using the interface. This was recorded on the same Camtasia video, using a microphone plugged into the computer. In addition, the comments entered were automatically stored on the server, so at the end of the session we copied the comments and deleted them from the server.

After completing the test tasks we asked the testers what they thought of the interface, using the post-test questionnaire. We used a similar questionnaire to collect their assessment of the control task, completing a paper questionnaire.

Finally, we invited the testers, in a group, to discuss their experiences in expressing their views through the computer map and the paper questionnaire. We took notes of points raised.

Designing a usability test

2008-01-13T17:38:57Z

Jjh:

We set out to compare the usability of this map-based interface with the conventional paper
questionnaire produced by the [http://www.pbni.org.uk/ PBNI].

==Selecting the testers==
There is no point getting students to test the interface since what we wanted to know was whether ex-offenders, some with low literacy levels, found the map-based interface more or less usable than current consultation techniques.

===NIACRO===
We needed to recruit ex-offenders to test the system. The [http://www.niacro.co.uk/ Northern Ireland Association for the Care and Resettlement of Offenders (NIACRO)] runs IT courses for ex-offenders and others. They have a small computer room in which trainees can develop IT skills that may increase their employability. A number of probationers take these courses. So [http://www.niacro.co.uk/ NIACRO] arranged for some of their trainees to test the interface when they came in for their regular session on 13 April 2006.

==Selecting the tasks==
We designed a sequence of tasks that started with simple familiarization with the map-based
site, then got progressively more involved as the testers gained confidence, until they were
entering comments on the consultation topic.

In brief, the sequence of tasks were:
* Explore the [http://www.pbni.org.uk/ PBNI] [http://www.pbni.org.uk/index/publications/consultation.htm e-consultation website].
* Find your probation office or reporting centre on the map.
* Find the all offices or reporting centres you have been to, and add comments on each place.
* Find your new office and reporting centre, and comment on how it would affect you.

The control task was to complete as much of the [http://www.pbni.org.uk/ PBNI] questionnaire as they could in 15 minutes.

==Collecting test data==
Before the tests started, we asked testers to complete a questionnaire on their familiarity with
computers, the Internet and consultations.

We set up [http://www.techsmith.com/camtasia.asp?CMP=KgoogleCStmhome Camtasia Studio], a computer activity recorder, on one PC. This software produced a video of the screen, including the position of the mouse at any time, and what the tester saw.

The tester spoke aloud during the test, explaining what the user was doing, and noting any difficulties in using the interface. This was recorded on the same Camtasia video, using a microphone plugged into the computer. In addition, the comments entered were automatically stored on the server, so at the end of the session we copied the comments and deleted them from the server.

After completing the test tasks we asked the testers what they thought of the interface, using the post-test questionnaire. We used a similar questionnaire to collect their assessment of the control task, completing a paper questionnaire.

Finally, we invited the testers, in a group, to discuss their experiences in expressing their views through the computer map and the paper questionnaire. We took notes of points raised.