It is against this background, of a developing IT culture within Europe and latterly the rest of the world, that the research with AEGON UK Services is set. Will the experiences outlined above be matched within the Company?
4. Research
‘Most people associate the word ‘research’ with activities which are substantially removed from day-to-day life and which are pursued by outstandingly gifted persons with an unusual level of commitment’, suggested Howard and Sharp (1983:6). However, Denscombe (1998:1) takes an alternative approach when opening his piece of work some 15 years later, ‘Social research is no longer the concern of the small elite of professionals and full-time researchers. It has become the concern of a far greater number of people who are faced with the prospect of undertaking small-scale research projects as part of an academic course or their professional development.’ In my case, the piece of research was to be carried out over a 3 month period by the Training Manager of a large organisation with very ordinary skills and an extremely heavy workload!
Payton (1979:4) identified research as ‘…the process of looking for a specific answer in an organised objective reliable way’ and it is this search for answers that has driven me to tackle the challenge of evaluating the European Computer Driving Licence (ECDL) in AEGON UK Services. Considering what questions to ask, Black suggests (1993:24/25), may prove a significant challenge in itself, ‘The most difficult part of starting a research project is often that of identifying the best question to ask, one that is meaningful, whose answer contributes to the discipline, and whose resulting research can be carried out within the resources available.’ When considering the questions to answer from the piece of research I was conscious of two things set down by Managing Director, David Barker: a) what would add value to the business supporting AEGON UK Services as a key part of the AEGON UK operation and b) provide learning opportunities to develop staff and upgrade their knowledge and skills? These factors contributed directly to my questions:
How has ECDL added value in AEGON UK Services?
How have the learners acquired new skills to help them do their jobs?
Has the method of learning for ECDL proved effective?
What is the most effective way for any further learners to acquire the knowledge and skills necessary to achieve ECDL?
Blaxter et al (1997:58) identified 2 different methods for designing and doing a research project, their ‘research families’ of qualitative and quantitative methods.
To Burns (2000:3), the quantitative approach is viewed as ‘scientific’ and in his opinion, ‘…has been the conventional approach to research in all areas of investigation. The methods and purposes of scientific inquiry have been moulded by countless generations of scientists …’ Burns (2000:9) outlines the strength of the quantitative approach as being its ‘…precision and control’, together with ‘…a deductive approach and the use of quantitative data (which) permits statistical analysis.’
Denscombe (1998:177) supports Burns’ standpoint by adding that, ‘The use of quantitative data in social research has its attractions. For one thing, it carries with it an aura of scientific respectability. Because it uses numbers and can present findings in the form of graphs and tables, it conveys a sense of solid, objective research.’ However, an alternative view also exists and caution is stressed by Silverman (2000:6) when pointing out that ‘ …the hard data on social structures which quantitative researchers claim to provide can turn out to be a mirage.’ Caution was necessary for me when considering the effect of ECDL upon an individual’s ability to do their job more effectively, as Cicourel (1964) points to defects with quantitative research, it may be useful but it also may conceal as well as reveal social processes.
On the other hand, qualitative research is an umbrella term that covers a variety of styles of social research, drawing on a variety of disciplines. Tesch (1990) identified 26 distinct kinds of social research which can fall under the term ‘qualitative’ and no doubt, in time, that list will grow. However, Denscombe (1998:207) does see some common elements which begin to give some sense, as he sees it, to qualitative research, ‘…a concern with meanings and the way people understand things…’ and ‘…a concern with patterns of behaviour….’
Cohen and Mannion (1998:8) go a stage further and view qualitative research as a search for understanding in which ‘…the principal concern is with … the way in which the individual creates, modifies and interprets the world in which he or she finds himself or herself.’ Researchers who adopted this qualitative approach, responsive to individual perceptions, were felt by Bell (1999:7) to ‘…seek insights rather than statistical analysis.’
A positive aspect of qualitative research, according to Miles and Huberman (1984:10), is ‘…that they focus on naturally occurring, ordinary events in natural settings, so that we have a strong handle on what ‘real life’ is like.’ The approach to this research project was on a qualitative basis and whilst there is a range of disadvantages to this method, including, as identified by Anderson (1990) – the ability to collect a large number of replies, allow for easy collation and cheap and readily available, Anderson did identify advantages with a qualitative approach. Significantly, when gathering data and information Anderson held that a qualitative approach enables the researcher to really understand another person, and qualitative research tends to focus on learners, central to the research project, and their views of the world. My own research project mirrors a number of Anderson’s findings; focusing on the learners and managers and their views, attitudes and feelings is key to my work and as such does not lend itself naturally to a quantitative or scientific basis.
However, in Denscombe’s opinion (1998: 173), the two approaches are not mutually exclusive, the distinction between the two is over-simplified and relates to the treatment of the data. As Strauss (1987: 2) argues, ‘…the genuinely useful distinction is in how data are treated analytically.’ Denscombe’s (1998:173) view then is that ‘…a distinction between qualitative and quantitative research is far from watertight.’
To some extent, the research design depends on what Barnes (1992:114) highlights as either a deductive or inductive approach to the project, with action research a key method of the latter. Barnes takes the view that when using action research, ‘…the outcomes…are generally increased knowledge, understanding and improved practice.’ The implication from this is that action research has a relevance for what I will be undertaking and indeed Blaxter et al (1997:64) confirm this by pointing out that ‘…it is well suited to the needs of people conducting research in their own workplaces, and who have a focus on improving aspects of their own and their colleagues’ practices.’
Cohen and Mannion (1998:186) also focus their thoughts on this work-related aspect of action research by suggesting that it ‘…is situational – it is concerned with diagnosing a problem in a specific context and attempting to solve it in that context.’ Although action research as a basis suits the needs of my project as shown above, there are a number of criticisms about the method, often reflecting Cohen and Mannion’s (1998:193) comments that it is not scientific, ‘…its sample is restricted and unrepresentative…its findings …are restricted to the environment in which the research is carried out.’ I accept the above but am satisfied that although my findings may be ‘restricted to the environment’, they are after all intended for use solely within AEGON UK Services, any wider usage will be a bonus.
Research data was accumulated by the use of questionnaires completed by learners, interviews with senior managers and Preston College staff, and focus groups conducted with groups of learners. This common multi-method approach is referred to as triangulation, as originally identified by the works of Elliott and Adelman (1976), and has been defined in Open University course 811 Study Guide (1988:54) as ‘…cross-checking the existence of certain phenomena and the veracity of individual accounts by gathering data from a number of informants and a number of sources and subsequently comparing and contrasting one account with another in order to produce as full and balanced a study as possible.’
The use of triangulation goes some way to meeting the need for this piece of research to be both reliable and valid. Bryman (1989:55) outlines reliability as something which ‘…refers to the consistency of a measure.’ Blaxter et al (1997:200) summarise the issue of reliability in simple terms as considering whether ‘…you have carried it out in such a way that, if another researcher were to look into the same questions in the same setting, they would come up with essentially the same results ….’ Validity is identified in a similar tone by the same authors, ‘…to do with whether your methods, approaches and techniques actually relate to, or measure, the issues you have been exploring.’ Triangulation is particularly important to me because it increases the strength and validity of my work.
The use of questionnaires is outlined by Denscombe (1998:88) who suggests that they are at their most productive when: used with large numbers, straightforward information required, the social climate is open, standardised, time allows for delays, resources allow for the costs and when the respondents can be expected to be able to read and understand the questions. However, my own piece of research is much smaller with fewer in the sample, straightforward questions on information that is current, all of which adds to the simplicity of this process.
However, there is a range of issues when using questionnaires with Bell (1999:75) suggesting that ‘It is harder to produce a really good questionnaire than might be imagined.’ McKernan (2000:125) endorses that view and warns that amongst the disadvantages are the amount of time taken compiling successful questions and low response rates. Nevertheless, the use of questionnaires provides direct access to the learners and adds balance to the interviews with my colleagues and members of Workplace Training Services.
Given that my questionnaire is straightforward with simple questions I am satisfied that this is a particularly useful method of gathering my research data. Further weight is given to this argument by Burns (2000:581) who supports me by adding such strengths as ‘…cost, each respondent receives same set of questions, errors in collation of responses reduced, respondents free to reply at own pace/time, fear and embarrassment avoided, may guarantee confidentiality.’ Burns also highlights a number of potential concerns with the use of questionnaires including a poor response rate when compared to interviews, the potential for bias due to poor returns and badly prepared questionnaires. I acknowledge Burns’ concerns and test my questionnaire on a small pilot group of learners, drawn at random, to assess its ease of use; 2 questions are reworded to reflect the comments of this group.
Given the small number of staff to have completed the ECDL programme I issue the questionnaire to all 45 which also supports the validity of my research and reduces the potential for any bias due to the choice of my sample. I am conscious that, as Cohen et al (2000:245) put it, ‘…the questionnaire will always be an intrusion into the life of the respondent, be it in terms of time taken to complete the questionnaire…or the possible invasion of privacy.’ As a result of this, great care is taken over the questions posed, the amount of content and the manner in which this is issued to learners. To keep the questionnaire simple I want to largely use closed questions and this method is supported by Wilson and McLean (1994:21) who confirm that these are ‘…simple to complete…and do not discriminate unduly on the basis of how articulate the respondents are…’. However, to ensure that people’s views are gathered I aim to allow respondents a small degree of space for their own thoughts to be considered which mirrors the views of Oppenheim (1992:115) when considering the use of closed questions: ‘…they do not enable respondents to add any remarks, qualifications and explanations to the categories, and there is a risk that the categories might not be exhaustive and that there might be bias in them.’ A blank questionnaire is included as Appendix B.
It is the need to gather personal views and opinions that prompts me to include interviews with senior managers, and focus groups amongst learners. As Denscombe (1998) points out, questionnaires are at their best with large numbers; I am dealing with 6 senior managers and am mindful that the culture of my own organisation is such that as Training Manager it is politic for me to see the senior managers rather than issue them with a questionnaire. As McKernan (2000:128) claims, ‘Interviewing is a social survey skill which can be taught.’ I am satisfied that my role in designing and delivering ‘Interviewing Skills’ training courses for 5 years to the same group of managers enables me to carry out this process!
There are a number of identified advantages in using interview techniques for gathering my research data and these include, Burns (2000:582/3) who is an exponent of its flexibility, ‘…the interviewer has the opportunity to observe the subject and the total situation in which they are responding…’ which according to Burns should produce higher response rates and ‘…if properly conducted should yield response rates of at least 80-85%.’ Cohen et al (2000:268) provides further thoughts and considerations on interviews, when outlining the need for trust to exist, suggesting that there should be a relationship between the interviewer and interviewee ‘…that transcended the research, that promoted a bond of friendship, a feeling of togetherness and joint pursuit of a common mission rising above personal egos.’ Once again, my relationship with the senior managers is such that I am confident that I have this level of trust in place already to enable a smooth process to occur. In addition, greater credibility is given to me conducting these interviews, as Kane (1997:68) asserts, ‘…the closer the interviewer is to the respondent in class, sex, age and interests, the greater chance the interviewer has of being successful.’
When considering the strengths and weaknesses of different types of interviews I endorse the comments of Patton (1980:206) who sets out the ‘interview guide approach’, which mirrors my requirements. The topic and issues to be covered are specified in advance, thus ‘…the outline increases the comprehensiveness of the data and makes data collection somewhat systematic for each respondent.’ I recognise that there is a risk of omitting important or salient points and that I must pay particular attention to maintaining focus on the key issues, in this case whether the managers consider that the ECDL programme has benefited the business.
Consideration however must be given to the school of thought that exists regarding problems with the interview as a method for gathering my research data. Mason (1997:42) suggests that ‘good qualitative interviewing is hard, creative work. It is a much more complex and exhausting task to plan and carry out … than, for example, to develop and use a structured questionnaire for asking a set of predetermined questions.’ Tuckman (1972) set out a number of problems with the interview as a technique for gathering data, including the fact that a competent interviewer is required (which may involve payment), a limited number of respondents can be reached and the reliability is quite limited. Whilst acknowledging these concerns, as I have set out my competency earlier, I require interviews with a controlled group of 6 individuals only and have a great degree of confidence in the reliability of my data. Not to mention that this interviewer requires no additional payment for the work.
My final information gathering technique is the focus group which Kreuger (1988:27) highlights as typically having 5 characteristics, ‘…a) people who, b) posses certain characteristics, c) provide data, d) of a qualitative nature, e) in a focussed discussion.’ Cohen et al (2000:288) support my use of the focus group by suggesting that they ‘…might be useful to triangulate with more traditional forms of interviewing, questionnaires, observation etc.’ Although observation is not necessary or appropriate for my research, Cohen et al do concur with my use of the interview and questionnaire.
According to the December 2002 issue of Croner’s A-Z Briefing (12/02:2), ‘…the main advantage of a focus group lies in its ability to collect subjective judgments where several, individual, judgments are better than just one. …By using a focus group …the researcher can see how and why individual judgments are supported ….’ This aspect of the focus group is particularly useful to me as it enables a group of learners to come together and share their experiences of the ECDL programme including highs and lows. It allows me the opportunity to facilitate discussion on the optimum way forward for AEGON UK Services and for future learners. Croner goes on to add that, ‘…they perform a useful function when evaluating various aspects of training and development, particularly when qualitative measurements are required and where it is deemed necessary to ascertain any unexpected outcomes or applications which have arisen as a result of such provision.’ Morgan (1988:43) cautions on the size of focus groups, suggesting ‘…between four and twelve…’ whilst Croner takes a similar stance, ‘…between 6 and 12…’
Both of the above agree on the need for careful consideration with sampling, in this case with the focus group, as Croner puts it (P3), ‘…all those involved do need to be representative of the total population under consideration.’ I have already decided to use all of my Award-holders as I have 3 months in which to carry out the research and have a small group, as Bell (1999:83) suggests, ‘The number of subjects in your investigation will necessarily depend on the amount of time you have.’ However, the question of the size of the sample and how it is selected is an issue.
Cohen and Mannion (1998:87) put forward a range of sampling tools including probability and non-probability, random and stratified samples whilst Bryman (1989:107) argues the case for a representative sample using a type of probability sample. To ensure that each of my ECDL Award-holders has an equal probability of inclusion in the sample used for the focus groups, names are drawn at random from an Excel spreadsheet. Kerlinger (1986:45), identifies randomness thus, it ‘… means that there is no known law, capable of being expressed in language, that correctly explains or describes events and their outcomes.’ Kerlinger (1986:44) goes on to provide further evidence that my sampling approach is correct, ‘… random sampling is that method of drawing a portion (sample) of a population … so that each member of the population … has an equal chance of being selected.’ Taking into account the thoughts of both Morgan and Croner above, I facilitate 2 focus groups of 8 members each.
There are some difficulties in using focus groups and some of these are identified by
Krueger (1988:46/7) who outlines the following: less control of the group as opposed to individuals, difficulties in analysing data, lack of interview skills, varying group dynamics, assembling groups and creating a supportive environment. I acknowledge Krueger’s observations and take extra care when running the focus groups although I must stress that my background is as a skilled interviewer and I am able to create a stable, supportive environment for the attendees who have attended similar workshops and courses in the past, often facilitated by me.
‘Any research project is likely to raise ethical issues. This is particularly so if it involves people directly, but may also be the case even if you conduct your research entirely on documentary evidence’, Blaxter et al (1997:146). Clearly, it is wrong for me to assume that, given my position as Training Manager, there is no issue with ethics, people know me and are happy to open up and provide me with valuable and valid information. Bell (1999:52) warns, ‘People will be doing you a favour if they agree to help, and they will need to know exactly what they will be asked to do, how much time they will be expected to give and what use will be made of the information they provide.’ The view is endorsed by McKernan (2000:241), who adds that, ‘It is crucial for all participants to know what their rights are in research of any kind.’
As such, I take great care when contacting my ECDL learners to alert them to my research and the need for their valuable input to further the learning partnership within AEGON UK Services. Particular attention is paid to the senior managers who give me some of their valuable time for 1:1 interviews and each of them is contacted personally by telephone to discuss the needs and requirements of the research and agree suitable times for the interviews. Hopkins (2000:221/2) identifies a range of principles that I must observe in the commitment to my research including, observing protocol, involving participants, reporting on progress, maintaining confidentiality and obtaining authority before using quotations. Particular care was observed when considering the use of quotations from our external partner, Workplace Training Services, and I arranged a special meeting with Cyril Wheat to discuss the learning partnership and seek his agreement to the use of specific quotations, included in the Conclusions to this research.
As Burns (2000:22/3) summarises, ‘All in all it looks fairly difficult to conduct much research without running into ethical arguments.’ Burns’ comments alert me to the need to take nothing for granted in view of my ‘special’ position with the learners, ‘Ethical problems are likely to occur in social science research since human subjects are involved. Researchers must be aware of ethical considerations involved in voluntary and non-voluntary participation, deception, informed consent, privacy and confidentiality, the right to discontinue, and obligations of the experimenter.’
It is this ‘special’ position, that of someone who has worked very closely with most of the ECDL learners, that Sellitz et al (1962:583) might have been thinking of then they attested, ‘… interviewers are human beings and not machines and their manner may have an effect on respondents.’ Many factors can influence responses one way or another, for instance Borg (1981:87) highlights a few of the problems that can occur, ‘… eagerness of the respondents to please the interviewer, a vague antagonism … or the tendency of the interviewer to seek out the answers that his preconceived notions ….These factors are called response effect by survey researchers.’ ECDL learners know me well and are aware of my enthusiasm for the learning partnership and our work with the ECDL programme. Might this naturally influence their responses to questionnaires, focus group or, even senior managers, interview questions?
However, Bell (1999:139) offers words of wisdom, just as the aspiring researcher wilts under the strains of potential bias, by pointing out that ‘… it is easier to acknowledge the fact that bias can creep in than to eliminate it altogether. This is a key consideration for my research as I am very keen for it to succeed being the instigator of the learning partnership with Workplace Training Services and the person responsible for the ECDL programme.
A further area for my research to draw upon is the Training team’s entry for the 2002 National Training Awards. The entry was based upon the team’s work with the ECDL programme in AEGON UK Services and I refer to the comments of the judges in the Data Analysis and Conclusions.
5. Analysis of Data
My research amongst European Computer Driving Licence (ECDL) award-holders in particular highlights the very positive aspects of the programme and it is encouraging to learn that all questioned feel that the programme has benefited them, personally and professionally. Significantly, the great majority of learners also felt able to now support colleagues with IT-related enquiries, a factor that reflects our status as an Investor In People; staff are keen to share knowledge with each other in a positive manner.
‘Being able to solve problems,’ ‘increasing one’s confidence’, ‘developing trouble-shooting skills’ and appropriately ‘staff trained in new technology’ are benefits to the company, as viewed by the learners. Although it is not the intention in my research to provide statistical analysis, rather to draw conclusions from learners’ responses, the feedback from learners does represent universal support for developing skills to enhance the business. Learners feel more competent to do their work, have a more confident approach and have developed skills to enable new work to be handled also converting manual tasks to a PC-based system. I am able to speak from personal experience and, as a result of taking the ECDL programme, I can now use Excel with some success. Whilst it is very difficult to make any claims with the research I am confident that the programme has made a significant difference. Prior to starting the ECDL programme I was unable to use Excel despite having access to the program, after completing the Excel module I am now able to design spreadsheets for use in the Training team.
There is evidence of other organisations introducing a successful ECDL programme and the following case study extracts are from the ECDL’s own website, . Mike Sampson, Human Resources Director at Royal Liver Assurance, commented: "We are delighted that the ECDL pilot scheme has proved to be so successful and now intend to run ECDL again with further members of staff. It is important that our employees are given the opportunity to refresh and improve upon their IT skills, which ultimately reflects a great benefit to Royal Liver and ensures the company's values and good standing are continually maintained." (www.ecdl.co.uk/employer/case/royal)
Even in the world of IT itself, ECDL is viewed as a positive addition to the learning curriculum, Les Williamson, Resources Servicing Manager, IBM, added, ‘We realised fairly quickly that this certification would be beneficial across the site. Not many people can claim proficiency in all the IT productivity tools that we could be using everyday. This is a qualification that demonstrates just that. (www.ecdl.co.uk/employer/case/ibm)
90% of learners feel that they are now more confident when using a PC (66% of the remaining 10% already did so) which is an endorsement of our decision to offer the programme and a statistic from a similar exercise conducted amongst award-holders at Manchester NHS Health Authority in 2002 found that ECDL qualified staff are proven to save an average of 38 minutes per day. Staff within AEGON UK Services also highlighted the speed at which they are now able to process IT-related work as a benefit to the Company.
Feedback from the learners in the questionnaires suggested that, whilst staff were achieving success using the CD ROM made available by Workplace Training Services, there was less than total satisfaction. Accordingly, amongst the questions I posed at the focus groups was ‘What method of learning would you recommend for future ECDL students?’ Although only a small percentage of respondents to the questionnaire felt that the quality of learning materials was less than ‘good’ during the focus groups, the mood was very different and people had the opportunity to express their views in more detail. It became apparent that the majority of the attendees of twelve felt that the CD ROM was difficult to work with, either because of their own IT situation out of the office, some did not have a PC at home, or more commonly because of the great amount of detail included. Their preferred learning method was to use paper-based material although some members of the focus groups felt that the CD ROM was ideal.
Comments were also made that it was often difficult to obtain 1:1 support as members of the Training team were often involved in other projects and, in truth, were never seen as providing close support for learners. Learners’ comments were referred to Cyril Wheat who visited the site and spoke with groups of the learners; as a result, directly, of the feedback provided to Cyril his Workplace Training Services team started to provide future learning support material in two forms, paper-based for the majority, CD ROM where preferred. In addition, another change in policy as a result of feedback, with the introduction of a dedicated resource from the College, someone who would be guaranteed to be on the Lytham site once a week for personal tutorials.
Most surprising amongst the findings was the feedback from the senior managers. Whilst the Managing Director has given the partnership with Workplace Training Services his blessing and has in fact recently enrolled on the ECDL programme, the same positive views are not held by the management team. I conducted the interviews and was surprised by the little knowledge that the managers had of the programme, one actually admitted that he had no awareness that any of his staff are taking an award. Whilst the general view amongst the managers was that any learning which develops the staff to enhance the business, must be a good thing, only one member of the team was more positive. One manager had experienced the ECDL programme personally, achieved the award, and spoke highly of the benefits to AEGON UK Services in supporting the implementation of COE. More work remains to be done with the management team for the future of the programme.
6. Conclusions
The comments of learners who have achieved their European Computer Driving Licence (ECDL) are extremely encouraging from a purely personal point of view, the initial thoughts are that my efforts in bringing the ECDL programme to AEGON UK Services staff has paid off. But that cursory observation is not sufficient to merit the programme a success.
When we set off on the ECDL programme there was a clear business need to be met, successfully implementing the Common Operating Environment (COE) and the transfer to Microsoft Office 2000. What must not be overlooked was that the business was not using any formal training strategy to meet the requirements generated by the COE and I was confident that ECDL could provide many of the training needs. Although I am not able to provide any hard quantitative data I have feedback from learners and managers which supports the use of the programme. As one manager put it, ‘Having achieved the ECDL I can vouch for the skills and knowledge acquired. Members of my team have demonstrated the benefits of the learning through the enhanced use of different programs.’ The data provides evidence of ECDL supporting the implementation of COE in the business.
New skills have been acquired by learners, many of whom are now looking for further opportunities, whilst at the same time encouraging colleagues to enrol on the programme. There are numerous examples quoted of staff who are now able to access different programs, use PowerPoint, Access or in my case, get to grips with Excel for the first time.
Our relationship with Workplace Training Services has developed, from the first tentative enquiries regarding the use of ECDL, the College’s first learning partnership in the North West to one of collaboration on other projects; as I close this research we are discussing certificated learning to meet behavioural competencies. Cyril Wheat, Manager Dedicated Delivery, is proud of his organisation’s role with the business and speaks highly of our work, ‘with the commitment of an employer who is providing resources is very important.’ The need for such support is echoed by Tim Rush of Islington Council who adds, ‘We have been lucky here in Islington, Members of the Council and senior management have been fully supportive. Gaining support at that level is essential to any successful implementation.’ (). Cyril’s team’s commitment to us now includes the use of 10 PCs, available for any form of learning, and a dedicated resource on-site for work with the ECDL programme.
Significantly, the learning has largely taken place at no cost to the business and during 2002 over 100 learners were enrolled with Workplace Training Services for free and we have ascertained that open learning with the ECDL programme may yet prove to be extremely cost effective to the business. We have 150 staff achieved or working towards the award with 100 more to follow during the course of this year an costs incurred to-date are a little over £3,000 which is approximately £12 per learner. Given that this learning ultimately accredits staff members with a universally accepted IT qualification and attendance on external learning is in my experience over £100 per day then I believe that this equates to value for money learning.
Our work with the fledgling ECDL programme in employer-related learning has highlighted new areas for study. As demonstrated earlier, learners had difficulties with the Computer Based Training and the CD ROM was not viewed as user-friendly; as a result, their feedback has been taken into account and staff enrolling more recently have had access to paper-based support material with CR ROMs available where required.
The judges at the 2002 National Training Awards were ‘… impressed by your decision to encourage your employees to undertake the ECDL qualification’ and drew upon a number of other positive aspects although they highlighted a number of important issues for the team to consider. As I point out above, there are limitations with my research and issues to consider for the coming months, as the judges point out in their feedback, ‘… it is still quite early in the process of introducing the Common Operating Environment’ and they have suggested that we ‘re-enter at a later date’.
I am also conscious that the research has been carried out by someone with a clearly vested interest in its success, the training of staff in the business is after all my responsibility, and that this may have clouded the findings. I acknowledge this limitation and make efforts to minimise the effects of this bias. My ultimate findings are also effected by yet more bias; the results are based upon my work with learners at AEGON UK Services in a supportive environment where facilities are made available to them and they do not have to pay any fees. I make no claims about applying my results to a wider audience whether in another industry or with the public at large.
However, the final thought lies with Andrew Mayo, writing in the January 2003 edition of Training Journal, who holds out, ‘So here is the challenge for 2003. Will you be able to say, at the end of the year, where and how learning interventions made a difference to the achievements of the organisation(s) you work with ….’ From the feedback received from learners and indeed Workplace Training Services’ staff I am confident that the ECDL programme has already made an impact on AEGON UK Services although greater communication is required with senior managers based upon their own feedback. Furthermore, as the judges of the National Training Awards indicate, there is yet more to come as our learning partnership matures and more staff set out on the road to success through the challenge of ECDL.
Project Proposal – DHL0730 Empirical Study
AEGON UK Services in partnership with Workplace Training Services, an arm of Preston College, started to offer the European Computer Driving Licence (ECDL) to staff in September 2001. The ECDL programme based it is upon Microsoft Office 2000 applications was seen as an ideal complement to the Common Operating Environment (COE) to be launched in 2002. At the same time the launch of ECDL provided further evidence of the Company facilitating learning and development opportunities for its staff. Under COE the whole of AEGON UK would operate with Microsoft Office 2000 whilst over 90% of staff were familiar with the 1997 version or earlier. To-date over 150 staff have started or finished the ECDL programme yet no evaluation of the success or otherwise of ECDL within AEGON UK Services has ever been carried out.
Title of Research
An evaluation of the European Computer Driving Licence (ECDL) as used by AEGON UK Services.
Brief Description
The ECDL was launched to develop the skills of European citizens and to enable them to become proficient in PC skills. The use of ECDL within AEGON UK Services was designed to provide staff with the skills necessary to deal effectively with the Common Operating Environment (COE), in effect Microsoft Office 2000.
Key Factors in Relation to:
Learners
To justify that the learners receive appropriate learning material.
To ensure that the needs of learners are catered for.
Company
To ascertain whether the ECDL programme actually met the original objectives of providing learning for staff members to ensure that they would have the necessary skills to operate Microsoft Office 2000.
Trainers
To consider what the trainers might learn upon reflection of the experiences of our original learners to maximise for the benefit of future ECDL candidates.
Questions
How has ECDL added value in AEGON UK Services?
How have the learners acquired new skills to help them do their jobs?
Has the method of learning for ECDL proved effective?
What is the most effective way for any further learners to acquire the knowledge and skills necessary to achieve ECDL?
Role of the Participants and Researcher (Trainer)
Sample group of learners to complete questionnaire on preferred learning styles.
Sample group of learners to take part in open focus groups, entitled ‘What have you learned and put to use from ECDL’?
1:1 discussions to take place with 6 Senior Managers on evaluation of ECDL.
Discussions to take place with Workplace Training Services’ staff to ascertain thoughts on effectiveness of different methods of learning.
Sample Groups
Conduct questionnaire research with staff who have completed ECDL
Random sample of 16 staff who have completed ECDL to be invited to focus group meetings, names drawn at random from Excel spreadsheet
Feasibility
Discussions to take place with Workplace Training Services re ECDL programme in other workplace environments.
Consider time constraints upon staff members, Senior Managers, Workplace Training Services staff and colleagues.
Timescale
Research to be completed in 3 months, as follows:
October: Research Plan drawn up and agreed with course leader
November: Research carried out
December: Evaluation
Data Collection Techniques
Interviews – Workplace Training Services staff, Senior Managers
Questionnaire - Learners
Focus Groups: Learners
Methods of Analysing Data
Annotate key issues from interviews; reflect back to interviewees to check accuracy
Compile charts of results of questionnaire findings.
Create graphs of results of learning styles feedback.
ECDL Evaluation Questionnaire
ECDL Evaluation Questionnaire
Bibliography:
Anderson G. (1990), Fundamentals of Educational Research, London: The Falmer Press
Cicourel A. (1964), Method and Measurement in Sociology, New York: Free Press
Hopkins D. (2000), A Teacher’s Guide to Classroom Research, Buckingham: Open University Press
Tesch R. (1990), Qualitative Research: Analysis Types and Software Tools, Basingstoke: Falmer Press
Tesch R. (1993), Software for Qualitative Researchers: Analysis Types and Programme Capabilities, in Fielding N. & Lee R. (Eds), Using Computers in Qualitative Research, (2nd Edtn), London: Sage
Tuckman B. W. (1972), Conducting Educational Research, New York: Harcourt Brace
References:
Barnes R. (1992), Successful Study for Degrees, London: Routledge
Bell J. (1999), Doing Your Research Project, (3rd Edtn), Buckingham: Open University Press
Black T. R. (1993), Evaluating Social Science Research, An Introduction, London: Sage Publications Ltd
Blaxter L., Hughes C. & Tight M. (1997), How to Research, Buckingham: Open University Press
Borg W. R. (1981), Applying Educational Research: A Practical Guide for Teachers, New York: Longman
Bryman A. (1989), Research Methods and Organization Studies, London: Routledge
Burns R. (2000), Introduction to Research Methods, (4th Edtn), London: Sage Publications Ltd
Cohen L. & Mannion L. (1998), Research Methods in Education, London: Routledge
Cohen L., Mannion L. & Morrison K. (2000), Research Methods in Education, (5th Edtn), London: Routledge
Croner Training and Development, A-Z Briefing, (No 52, 12/02), Focus on …Focus Groups
Denscombe M. (1998), The Good Research Guide for Small-Scale Social Research Projects, Buckingham: Open University Press
Elliott J. & Adelman C. (1976), Innovation at the Classroom Level: A Case Study of the Ford Teaching Project. Unit 28, Open University Course E203: Curriculum Design and Development, Milton Keynes: Open University Educational Enterprises
Finegold D., & Soskice D.(1988) The Failure of Training in Britain: Analysis and Prescription, Oxford Review of Economic Policy, 4, (3), pp. 21-53
Howard K. & Sharp J. A. (1983), The Management of a Student Research Project, Aldershot: Gower
<http://ecdl.com/main/index.php>, European Computer Driving Licence
<http://www.electricpaper/products/ecdl.asp>, The Electric Paper Company Ltd
IT Training, October 2002 P7 & June 2001 P7
Kane E. (1997), Doing Your Own Research, Marion Boyars: London
Kerlinger F. N. (1986), Foundations of Behavioural Research, (3rd Edtn), New York: Holt, Rinehart and Winston
Krueger R. A. (1988), Focus Groups, A Practical Guide for Applied Research, Beverley Hills: Sage Publications Ltd
Mason J. (1997), Qualitative Researching, London: Sage Publications Ltd
Mayo A. (2003), ‘I am your Business Partner?’, Training Journal, (January 2003:P36)
McKernan J. (2000), Curriculum Action Research, London: Kogan Page Ltd
Miles M. & Huberman A. (1984), Qualitative Data Analysis, Beverley Hills: Sage Publications Ltd
Morgan D. L. (1988), Focus Groups as Qualitative Research, Beverley Hills: Sage Publications Ltd
Open University Course 811 (1988), Educational Evaluation, Milton Keynes: Open University Educational Enterprises
Oppenheim A. N. (1992), Questionnaire Design, Interviewing and Attitude Measurement, London: Pinter Publishers Ltd
Patton M. Q. (1980), Qualitative Evaluation Methods, Beverley Hills: Sage Publications Ltd
Payton, O. D. (1979). Research: The Validation of Clinical Practice, Philadelphia: F.A. Davis
People Management (2000), ‘UK Skills to run training awards’, Vol. 6, No 25, p.8
Reay D. G. (1994), Evaluating Training, Kogan Page: London
Sellitz C., Jahoda M., Deutsch M. & Cook S. W. (1962), Research Methods in Social Relations, (2nd Ed), New York: Rinehart and Winston
Silverman D. (2000), Doing Qualitative Research, A Practical Handbook, London: Sage Publications Ltd
Strauss A. (1987), Qualitative Analysis for Social Scientists, Cambridge: Cambridge University Press
Wilson N. & McLean S. (1994), Questionnaire Design: a Practical Introduction, Newtown Abbey: University of Ulster Press
