Friday, July 30, 2010

Methodology Matters in Digital Learner Research

The study by Hargittai et al. (2010) that I reported on in my previous post is and example of a methodologically sound piece of research. This is unlike much of the popular writing on this topic which tends to be speculative, anecdotal or research based on questionable methodology. Educators have tended to pay more attention to the latter than the former. The result is a widepread belief that all people born after 1982 are technologically sophisticated and digtially literate and that we need to make radical changes to our educational systems to accomodate this generation. As the growing body of sound research, like that of Hargittai et al. (2010) is showing, this popular view is not grounded in fact.

Here are some key methodological strengths of the Hargittai study:
1) they actually observed user behaviour rather than relying on self reports and they didn't restrict what Web sites participants could consult: "Our findings suggest that utilizing this more naturalistic method allows us to uncover user practices that have been hard to capture using earlier methods."
2) They linked trustworthiness and credibility to branding which has been neglected in earlier studies and they took into account the full search context in their investigation: "How users get to a Web site is often as much a part of their evaluation of the destination site as any particular features of the pages they visit."
3) They used a mixture of quantitative and qualitative methods.
4) They administered a paper/pencil survey, "to avoid biasing against people who feel less comfortable filling out Web forms or who spend less time online and thus may have less opportunity to participate."

Look at the popular Net Gen literature and you won't find this kind of methodological rigour. And you won't find this conclusion:
"Students rely greatly on search engine brands to guide them to what they then perceive as credible material simply due to the fact that the destination page rose to the top of the results listingsof their preferred search engine."

A British study released last year came to similar conclusions about the information seeking behaviour of what it called the "Google Generation", a slighter younger group born in 1993 and later.

Thursday, July 29, 2010

The Digital Literacy of "Digital Natives"

A clear trend is emerging in the net gen literature. On one hand, the pundits and futurists continue to argue that "digital natives" are sophisticated users of the new technologies who critically analyze the information they access online. In most cases, we have to take their word for these claims because the underlying research (if there is any) is often proprietary and the authors reveal little of their methodology.

On the other hand, most of the academic research on this topic is showing that generation really isn't an important discriminator and that "digital natives", in fact, appear to have a superficial understanding of the new technologies, use the new technologies for very limited and specific purposes, and have superficial information-seeking and analysis skills. Now a new study has just been published that provides further evidence of the need to be extremely skeptical of the the often-repeated claims made by the likes of Tapscott, Prensky, and Palfery & Gasser and others.

Hargittai, Fullerton, Menchen-Trevino and Thomas (2010) investigated how young adults at a US university look for and evaluate online content. They found that the students they studied displayed an inordinate level of trust in search engine brand as a measure of credibility: "Over a quarter of the respondents mentioned that they chose a Web site because the search engine had returned that site as the first result suggesting considerable trust in these services. In some cases, the respondent regarded the search engine as the relevant entity for which to evaluate trustworthiness, rather than the Web site that contained the information." Only 10% of the students bothered to verify the site author's credentials: "These findings suggest that students' level of faith in their search engine of choice is so high they do not feel the need to verify for themselves who authored the pages they view or what their qualifications might be."

When asked how they decide to visit a Web site, the most important factor mentioned by the students was "being able to identify easily the sources of information on the site". However,  "knowing who owns the Web site" and "knowing what business and organizations financially support the site" were less important to students. When asked how they determine the credibility of the information, the least common actions were "checking if contact information is provided on the Web site" and "checking the qualifications or credentials of the author." Checking the "about us" section the Web site was also something that students did either rarely or on average.

Contrast these findings with what Tapscott (2009) has to say in Grown Up Digital:
"Net Geners are the new scrutinizers. Given the large number of information sources on the Web, not to mention unreliable information - today's youth have the ability to distinguish fact from fiction. The Net Generation knows to be skeptical whenever they're online. "

Palfrey & Gasser (2008) in Born Digital provide a slightly different, but equally positive, perspective on the critical faculties of "digital natives". They argue that "digital natives have a sophisticated process for gathering information from the Web that allows them to develop a deep understanding of current events and issues is often underestimated." "Digital natives" are "interacting with information in constructive ways", gathering information "through a multistep process that involves grazing, a 'deep dive', and a feedback loop. They are perfeting the art of grazing through the huge amount of information that comes their way on a daily basis."

But here's the important difference between the work of academic researchers like Hargittai and colleagues and books by people like Tapscott and Palfrey and Gasser: Hargittai's work has been subjected to peer review by experts in the field, has been published in academic journals and provides full details of the research methodology and how the research was funded and supported.

As Hargittai and colleagues conclude:
"While some have made overarching assumptions about young people's universal savvy with digital media due to their lifelong exposure to them (e.g., Prensky, 2001; Tapscott, 1998)...empirical evidence does not necessarily support this position...Students are not always turning to the most relevant cues to determine the credibility of online content."


Read the full article, Trust Online: Young Adults' Evaluation of Web Content.

Monday, July 5, 2010

What's the Big Deal? - Part 2

When a former president of Columbia Teachers College starts repeating the net gen myths, I start to get worried. That's what Arthur Levine has done in a post to Inside Higher Education,  Digital Students, Industrial-Era Universities. In it, Levine repeats all the same old claims about "digital natives": they "live in an anytime/anyplace world, operating 24 hours a day, seven days a week, unbounded by physical location; they are "more concerned with the outcomes of education — learning and the mastery of content, achieved in the manner of games"; they are "active learners, preferring interactive, hands-on methods of learning such as case studies, field study and simulations"; they "are gatherers, who wade through a sea of data available to them online to find the answers to their questions"; they "are are oriented more toward group learning, multiple “teachers” or learning resources, and social networking, characterized by collaboration and sharing of content."

Based on these unfounded claims, he then argues that higher education should be making significant changes to the curriculum, pedagogy, and delivery methodologies: "What must change...is the means by which we educate the digital natives who are and will be sitting in our classrooms — employing calendars, locations, pedagogies, and learning materials consistent with ways our students learn most effectively. "


I don't have a particular issue with any of the proposed changes that Levine puts forward. Most address issues of flexibility and responsiveness and moving from a teaching to a learning paradigm and away from common processes to common outcomes. As Levine says, "with this shift will come the possibility of offering students a variety of ways to achieve those outcomes rooted in the ways they learn best." Yes, but grounding those changes in unfounded and stereotypical views of a generation is dangerous because, as I have said before, all the credible research (ours included) suggests this isn't a generational issue and that viewing it in those terms hides more important differences.

Another example of why debunking the Net Gen myth is a big deal.

Sunday, July 4, 2010

The Challenges of Constant Connectivity

In my previous post I highlighted Neil Selwyn's EDMEDIA debate presentation in which he argued that too much of the social media in education discourse is happening within an "ed tech bubble". He referred to the "self-contained, self- referencing and self-defining nature of the debate" and he called for a broader discussion with people outside "the bubble".

I'm not sure if Sherry Turkle would be considered outside the ed tech community but she is certainly somebody who brings a refreshingly thoughtful perspective on the issues related to the use and impact of digital technologies.

Here is an edited version of an interview she gave for the excellent PBS series, Digital Nation, in which she talks about the challenges of constant connectivity and argues for a more considered approach to digital technologies that recognizes that we are in the early days of learning how to use these technologies and understanding their impact.


"I don’t really care what technology wants. It’s up to people to develop technologies, see what affordances the technology has. Very often these affordances tap into our vulnerabilities. I would feel bereft if, because technology wants us to read short, simple stories, we bequeath to our children a world of short, simple stories. What technology makes easy is not always what nurtures the human spirit."

Read: Digital Demands: The Challenges of Constant Connectivity

Friday, July 2, 2010

Social Media and Education

Neil Selwyn has written an excellent critique of the view that social media is having a postive impact on education. The Educational Significance of Social Media - A Critical Perspective is Selwyn's presentation at an ED MEDIA 2010 debate on the motion: "that the use of social media and networking is contributing to the attainment of significant educational goals in ways that suggest even more powerful future impact."

Selwyn argues that the social media discourse is "driven by belief, speculation, anecdote and personal experience rather than recourse to actual evidence."  He takes aim at a central part of the net gen discourse: the view that "digital natives" are actively engaged with social media, not just as passive consumers but active contributors. "The majority of people who do use social media are perhaps best termed as ‘non-active users’ – passively downloading content rather than engaging in any meaningful acts of creation or sharing."


He concludes by highlighting what I think is one of key problems with the ed tech/social media field today, the fact that too much of our discourse is self-referential and self-congratulatory, happening in an "Ed-Tech bubble." He says "we need to stop talking amongst ourselves, and start talking to those people outside of the educational technology community who do not usually engage in such discussions. One of the obvious limitations to current enthusiasms for social media is the self-contained, self- referencing and self-defining nature of the debate. These are generally conversations that only ever take place between groups of social media-using educators – usually using social media to talk about the educational benefits of social media. Outside of the narrow ‘Ed-Tech bubble’ very few people are engaging with these discussions. We therefore need to ... stimulate a new phase of discussion, dialogue and conversation about what social media is – and what social media could be – with everyone involved in education."

Other work by Neil Selwyn:
Adult Learning in the Digital Age: Information Technology and the Learning Society
The Digital Native: Myth and Reality

Tuesday, June 29, 2010

What's the Big Deal?

That's often a question I get asked about our research around the net gen and how learners are using new technologies for social and educational purposes. The implication being that it really doesn't matter if this net gen thing is overhyped and not very accurate. Well, here is an example of why it does matter.

The University of Missouri appears to have grounded its new online learning initiative on the misguided and ill-informed view that all learners in the net gen age group are technology addicts who constantly multitask, communicate digitally and want to use these technologies in their learning. "That's how they learn. That's how they think," says Steven Graham, senior associate vice president for academic affairs at the University of Missouri. Well, actually, it's not, at least not all of them. In fact we know very little about the impact of digital technologies on thinking and learning. What we do know is that all the credible research suggests this isn't a generational issue and that viewing it in those terms hides more important differences. There is a fair amount of survey data on student preferences for technology in learning and it tends to show that, in general, they prefer a moderate amount and face-to-face teaching is the preferred mode. As an example, the 2009 ECAR Study of Undergraduate Students and Information Technology concludes that students, "are more likely to describe themselves in terms of mainstream adoption of technology, and they consistently report that they prefer only a moderate amount of IT when it comes to their courses."

Despite that the University of Missouri is spending nearly half a million dollars to develop 124 new online courses because that's what they think students want.

That's why this is such a big deal.

Wednesday, June 2, 2010

Small Study Finds Gap between Educational and Informal Use

Only 29 students were surveyed in this study by Swapna Kumar so we have to be cautious about the results. The undergraduate education students at a large private university were asked about their perspectives on Web 2.0. technologies, specifically about their informal and educational use of these technologies.

The results confirm other studies that show that technology use is multifaceted and that we need to look more deeply at how the technology is being used. Like other studies, this study found students consumed far than they produced with technology. In other words they were primarily passive users of the technology and not making full use of the affordances of Web 2.0 technologies.

On the other hand, Kumar's finding contradicted other studies in finding that the students were able to transfer their personal expertise with the technologies to the academic context:

"the qualitative data summarized earlier in this paper reveal that students in this group use Instant Messenger when completing assignments, and Google Docs for archiving and group work, even if their professors are unfamiliar with Google Docs. They suggested innovative and relevant ways in which online videos, podcasts, and wikis can enhance their educational experience in contrast to research reported by Caruso and Kvavik (2005) and Kennedy et al. (2008). Students’ voluntary descriptions of how these resources have been used in courses that they have attended, as well as their enthusiastic suggestions, signify their interest in the use of new technologies in higher education."

But as Kumar points out, this interest in the academic use of the technology may have been driven by the students' interest in teaching and the fact they were enrolled in an educational foundations course. As we have said, the context is critical.


Tuesday, April 27, 2010

New Study Debunks Digital Native Myth...or does it?

One of my critiques of the net gen discourse is that it has been fuelled in large part by non-academic research. Many of the claims emerge from proprietary studies that have not been vetted through the academic peer review process. The authors of these studies are not required to disclose important details such as methodology, funding sources, and potential conflicts of interest. Without this kind of detail it is difficult to assess the quality of the research and the validity of the findings. Despite this, the conclusions are disseminated far and wide using social media and soon become entrenched.

Well, we have a new study that purports to show that digital natives aren't as technologically savvy as people like Tapscott and Presnky would have us believe. Our research certainly supports this conclusion but the trouble with this new study is that:
a) it was conducted by a private consulting company, Cengage Learning so, as far as I know, there was no requirement for peer review;
b) only very limited methodological details have been publically released.

Despite that, the headline that is appearing in the blogosphere is something like "Digital Native Myth Debunked". If we look at the data that this conclusion appears to be based on, it is pretty thin: "65 per cent of instructors think students are tech savvy when it comes to using digital tools in the classroom. Conversely, only 42 per cent of students believe there is enough support for educational technology, evidence of a perception gap in how adept students are versus how savvy they are presumed to be." Hardly myth-debunking evidence. What is more troubling is all we know about this study is that data was collected via a survey of 765 students and 308 instructors. We don't know where these instructors and students were, what the response rate was, how they were selected or what the actual survey questions are.

Saturday, March 20, 2010

Six Reasons to be Skeptical

Six reasons to be skeptical of the Net Gen discourse:

1. It exaggerates the gaps between adults and youth.
According to Henry Jenkins, adults are "seen as fumbling and hopelessly out of touch, and youth, seen as masterful". This encourages adults to feel helpless, and justifies "their decision not to know and not to care what happens to young people as they move into the on-line world." Ultimately it disempowers adults.

2. It hides more important intra-generational differences.
According to Reeves & Oh, research shows that "generational differences are weak as a researchable variable. " It also shows that differences in how learners use technology is often greater within an age cohort than it is between and that treating net generation learners as a homogeneous groups ignores these important differences. See Pedro (2009) and Kennedy et al. (2007) and (2008).

3. It ignores potentially important socio-economic and cultural differences.
Almost of all the claims about the net generation are based on observations of middle and upper class north American youth.

4. It ignores important second level digital divides.
By promoting the stereotype that all youth are sophisticated users of digital technology, the net gen discourse overlooks the inequalities in the capacity to use technology, skills and competencies required and information literacies. According to Thompson (2009)"Without attention to these potential second level digital divides, gaps and inequalities may widen over time despite concerted efforts to provide access to ICT"

5. It is based on unfounded assumptions about current approaches to teaching.
One of the key themes of the net generation discourse is that the current educational paradigm does not adequately deal with the needs of the net generation. The argument is that we need to move away from the current transmission mode of teaching to a more student-centered, interactive and collaborative mode. While there may still be a lot of this style of teaching, most public school education at the K-12 level moved away from the transmission mode over 20 years ago. In higher education we see the widespread use of case-based, problem-based, inquiry-based and experiential learning approaches.

6. The evidence doesn't support most of the key Net Gen claims.
Almost all of the claims of the net gen discourse are in popular media and if they are based on research, it is proprietary and full methodological details are not provided. All of the sound research that refutes the claims is published in scholarly journals and has been subject to peer review.

Tuesday, March 16, 2010

Researching the Net Generation: Separating Fact from Fiction

Elluminate Live presentation: March 17, 10 am PDT
Mark Bullen, Tannis Morgan, Adnan Qayyum

http://elluminate.nait.ca/join_meeting.html?meetingId=1261533799686

Description

Generation is often used to explain and rationalize the use of information and communication technologies (ICTs) in higher education. However, a comprehensive review of the research and popular literature on the topic and an empirical study at one postsecondary institution in Canada suggest there are no significant generational differences in how learners say they use ICTs or their perceived behavioural characteristics. The results of this investigation add to a growing body of research that calls into question the prevailing net generation discourse that argues that generation can be used to explain the use of ICTs in higher education. This presentation will review and analyze the key net generation claims and the growing body of empirical research that contradicts the popular view of the net generation learners as being sophisticated users of digital technologies with unique needs.