Anderson

From LTCWiki

Research Models for Connectivist Learning

Online Connectivism Conference

February 9, 2007

Transcript of a presentation by Terry Anderson

Abstract:

The development of any new pedagogical theory or practice depends upon high quality and systematic research to validate the theory and the resulting intervention's result and efficacy. Traditionally, education has imported medical and science models of quantitative study and social science and humanity tools such as survey's and qualitative investigation to substantiate and develop new pedagogy. However, there is increasing evidence that the context of real education and learning is too complex, idiosyncratic and culturally bound to benefit from the types of research used in other disciplines. Recently "design-based research" a methodology devised by and for educators has been celebrated as an effective way to study and develop new pedagogy. This presentation overviews design-based research theory and methodology, provides examples and overviews the critique of other research methodologies.

George: Welcome to our final day of the Connectivism Online Conference. It’s been a wonderful week of learning and sharing and forming connections with each other. I’d like to start by saying thank you to the Learning Technologies Centre here at the University of Manitoba. They’ve provided the hosting within the Moodle space and certainly the contribution of my time to this conference comes from them. I’d also like to say thank to Elluminate. I’ve heard some great comments about Elluminate as we’ve gone through the conference. A wonderful tool for again sharing and connecting with each other. And there is a slide I’ll demonstrate at the end where you can sign up for a free, three-person room if you want to use that just in your own personal collaboration, discussion, whatever you will. At this point I want to flip over, we’ll skip past a few of these slides here, to turn it over to Terry Anderson. For most of you, like many of the other presenters that we’ve had represented in the conference, they’re well known in their field or discipline. Terry is certainly no exception. He’s been a strong voice in our open learning space, he has contributed extensively from a research perspective, and also from contributing and formalizing some of the ideas we’re looking at through the online journal that he edits and as well as books that he’s contributed including open access or free download books. So certainly Terry has contributed enormously to our field, and I’m quite privileged to introduce him now to talk about some of the things we can think of for the future as we consider the research aspect of this space. So Terry, it is my pleasure to turn the microphone over to you.

Terry: Good, well, thank you, George. Thank all of you. It’s the largest Elluminate gathering that I’ve ever addressed, and I’m quite honored to be asked. I looked at the list of presenters and that’s real nice to be able to end it off, “bring it on home” as we say in Alberta. So I’m real pleased to be here. And also as I scan through the list of participants, I see some old friends and new friends, so welcome all.

I’m going to talk about research methods and probably more generically than the other presenters this week, this research talk can be relevant to almost any kind of learning beside connectivist.

I hope – I know there’s a few people who I’ve done portions of this talk before – I see their names listed and I’m getting embarrassed already, but I hope you’ll be patient and bear with me if I repeat some things that you’ve already heard. And I do hope that some of you are real brave and push the little hand up and ask a question. I can end up babbling on far more than I should or could. I will probably zoom through these slides and hope that you’ll enjoy them and be stimulated to get out and do some of your own design-based research.

You’ll see that I’m going to talk about research in general, education research in particular, then I’m going to talk a little bit about the major paradigms of research, and more or less put them up as straw dogs, talking about what’s wrong with them, and then I’m going to have the perfect solution, which of course is design-based research. So it will be a little bit of a plug for that particular methodology. So let me just zoom right along.

I guess I’d like you to put your thinking caps on for a moment and talk about what educational research has really made a difference for you as an educator or as a learner. I got asked this question in Hong Kong one time when I was on a panel at the International Council of Distance Ed. about what’s the one thing that really education research has contributed. I was blindsided. I though, here am I, big researcher, and I can hardly even think of one thing that has really made a difference. So I guess it’s of interest to know if you’re thinking about car research, or medical research, it wouldn’t be too hard to think of great cures or great engines or great airplanes or whatever, but it’s a bit strange, isn’t it, that we can’t think of anything that’s really made a difference in education research. Well, I shouldn’t speak for you, but you can tell me, and if you have thought of some particular research that has made a difference, try to dig down and say well, was it because it’s an interesting question, or was it because it’s the context that really was similar to where I teach or learn or work, or was it because it used a methodology that really resonated with you and sort of spoke some truth out of it, or was it because you heard about it and the clarity of the presentation in whatever media really hit you? So I’ll leave that with you to think about what your impression of education research is.

When I go to conferences and in too many of the papers that get submitted to the journal I edit, typical ed tech research is “here’s what we’re doing at my school and isn’t it wonderful” – a sense that we’re really innovative, on the cutting edge, and I want to tell you all about it. It doesn’t usually have a theoretical basis, the data collection is usually quite skimpy, and oftentimes people don’t realize that other people are doing the same thing and “build standing on the shoulders of giants” and that sort of thing.

The second thing and I get a lot of it, “here’s what we’re doing, why don’t you come and research it,” as if people who were at universities were just sitting around looking for places that they can research. And I think you get lots of variability by the journal, by the conference, by the region, by the topic. Ah, and somebody is talking about progressive inquiry, PBL, yes, I think that’s one of the pedagogies or learning activities that’s really taken off, and after 20-30 years of PBL research, especially in medicine, we’re finding that it really has had positive impact, and without that kind of research that’s been going on with PBL we might not think that. Good point.

OK, moving along. So what is research? Systematic study of materials and resources to establish facts and reach new conclusions. There’s some debate whether it has to be new – new to whom? New to you? New to everyone in the world? - to make it research. But you can see the kind of adjectives people use to describe it: organize, creative, systematic, scientific, diligent. Those kind of adjectives tend to – hopefully they inspire researchers, but they can scare me and practicing teachers because I think oh, man, it’s way too heavy duty, it’s way too much for me to be able to say that I can do all those things. So I think that’s one of our problems in education: we have a huge divide between the practitioner and the researcher, and trying to find a middle ground where these adjectives resonate for all of them is a challenge.

So why do we even bother doing connectivist research? Well, I think it’s fairly self-evident, but there are many unresolved questions of traditional education, non-connectivist, like efficiency, instructional design, how much does it matter whether you participate, what class or gender, it goes on and on and on. And of course all of those factors are carried over into learning or education or both when we talk about it from a connectivist perspective. We’re also finding that there’s all sorts of new forms of education provision – you know, when we can start mediating more and more of our stuff, when we pop into Second Life, when we talk about if this was a video conference and we could see each other, you could see my face, you know what would really make a difference – we don’t just don’t know a lot of the answers to these questions. Especially the bottom one – Web 2.0, social software – does it really encourage new forms of community, of inquiry, are they as effective, are they more effective than face-to-face learning? I think we just really don’t know the answers to those questions.

So why education research just don’t get no respect? Well, most research is not valued by the funders, other academics, or even by practitioners. I don’t know if you were to ask that question about the value of research or you ask – I know my brother’s a high school teacher. He thinks that research hasn’t done anything for him in the classroom ever and probably never will. If practitioners, who are supposed to be the consumers of our research, don’t value it, then we certainly are in trouble.

If you look at the statistics – these are Canadian, a few years old now – but in education it’s about 0.01% of what we spend on education do we spend on educational research. In medicine they have a goal of 3%. In high tech companies typically their R&D budgets are in the 10-15% of their total expenditures, and overall in Canada, it’s 1.88%. But look at the difference between the average 1.88% compared to education 0.01%. So we just don’t spend any money on education research. And you can ask yourself, why is that when we spend a lot of money on education? Why don’t we spend it on the research? It’s an interesting question, and I’ll put up a straw dog suggesting that our methodologies are part of the problem.

And typically, when people professionals look at educational research from other disciplines, they get slammed for not being rigorous enough, not controlling all the variables, making inappropriate generalizations, not thinking about all the complexities – the individual, the cultural, the linguistic, the environmental variables – not enough concern with teacher and tutor support, and a lack of really following up on – for instance, if you were to type in “connectivist research” into, say, Google Scholar, how many studies would you find? I bet you wouldn’t find - well, I guess somebody better try it and tell us, but I don’t think it would be very many: maybe only four or five, who knows.

OK, this is practitioner’s perceptions are too narrow to be meaningful, too superficial to be instrumental, too artificial to be relevant, and on top of that, they’re too late. So that’s our problem; what’s our solution?

Well, one of the – more on problems: it’s nobody’s job. Let me get you to use the little tool here and I’ll change the survey tool to a yes/no option and how many people of the 124 people in this conference, how many people at least 50% of your job is to do research? If it is, can you give me the green arrow, and if not can you give me the red x on the top, and I’ll reveal the statistics here. OK, there is eleven people who have 50% of their job doing research. There’s 54% people who aren’t answering, so we can double those numbers, so roughly 16% of the people here have 50% of their job. The number of times I’ve asked groups that, that’s by far the highest, so congratulations! Now I’m even more intimidated with my biased slides coming along. But in any case, you can see that it’s not a lot. We’re a select group, a self-selected group and we’re not all, but we do have 8% of us have 50% of their time doing research.

Research Paradigms

OK, so I’m going to quickly go over the four major research paradigms – as I see them, anyways. The quantitative, where we’re trying to discover all the laws that govern behavior. The qualitative, where we’re trying to understand from an insider perspective. The critical – I put Karl Marx up there because we’re trying to figure out who’s getting screwed and where the power relationships are in the phenomena we’re investigating. And then I’m going to end off with Ann Brown first talked about design-based research, where we’re talking about interventions and interactions and their effects in multiple contexts. And I’m going to focus and give an example of a design-based study that we’re in the middle of, because I think it’s got the most value for connectivist research.

OK, so, first, quantitative paradigm. You’ll find that it’s full of jargon, especially coming out of the States these days, the “no child left behind,” the fact that they have the “what works” site, they talk about evidence-based systematic review, they talk about research that’s “scientific” and that which is not, so there’s a real push by the quantitative folks to say that part of the problem with education research is that we haven’t done it like real researchers, and we can’t figure out what really works in all sites. And they use scientific discourse from positivism, realism, they basically take the medical model, or the natural science model, the work done in laboratories, and they try to extrapolate that and do it in classrooms. The gold standard of a quantitative standard: they always have to have a control group and if it’s any good, it will have random assignment. And the reason they do that is by randomly assigning a group of people to a control group and one or more treatment groups, they can see what’s different or what changes, and by controlling all the other variables, they can instantly find out what the truth is and what works. The problem is context. Context is so pivotal to education and there can be a zillion variables from whether the teacher is feeling well, whether the students had a fire engine drive by that day, to what the content is, to the temperature in the room, and it goes on and on and on and on. And top of having so many variables, they interact with each other, such that one will amplify and so that’s one of my biggest concerns with quantitative research is that I don’t think there’s one thing that works in all contexts. One way that you can try to get out of the mess of context is to do what they call a meta-analysis, where you gather the results from a whole bunch of different tests, hopefully with the same variables being tested and controlled, and then you can put them together, you get a much larger hand and make more valid comparisons. So that’s an overview of the quantitative paradigm. As I said, it’s very hot amongst funders, especially in the States these days. Right from Rene Descartes who says you can’t measure it, then it doesn’t really exist. That’s quantitative perspective.

I see somebody has their hands up, so I’m going to pause and scroll down to find out who just put their hand up and hopefully it wasn’t a mistake. I can’t even see their name here, I have too many names. Go ahead with the question.

George: Hi Terry, I think it might have been a mistake. I saw that as soon as the hand went up it went down right away, so it might have just been pressing the wrong button, so if you just want to continue.

Terry: OK, sorry about that. Let’s move along. Is meta-analysis the gold standard? And I have one Canadian example here that I wanted to talk about. This is done by Charles Ungerleider and a colleague of his at UBC, 2003. The question was, “Does information communications technologies in education make a difference?” Does it really help? And they were trying to look at a number of variables or outputs. You’ll find that all meta-analyses – that I’ve ever read, anyway – they always whine at the bottom: “Oh, there just wasn’t enough studies,” and “Oh, they didn’t control all the variables,” and “Oh, it was badly done.” That just comes with the turf. You always have to say that. And you always have to look – go through the literature and find 500-800 studies that were published, and then you have to say three-quarters of them were crap because they didn’t have control groups and they didn’t work and blah, blah, blah. But my real problem with Ungerleider and Burns’ study tends – I think there were 25 in total that met their criteria that they used. The type of interventions was incredibly diverse. They compared a full graduate-level course that was done completely online with no face-to-face versus some kids in a grade three class who did a Web safari for one afternoon, and then they’re trying to say this is ICT and what works and what doesn’t work. Well, the intervention itself was just so vastly different that trying to pull them together seems to me very problematic.

But here’s their results. You can see the study down on the left and what they were measuring: final marks or test scores. And this is what they call the effects size, which says you know was it a positive effect from using information communications technologies or was it a negative effect. And you can see on the results that some of them had negative effects, a few had positive effects, and when they put them together, lo and behold, no significant difference: 0.00. So what does it really tell us? It tells us that some ICT interventions in some classes have positive impact and some don’t, but does gathering them all together to show a 0.00, does it really help us? I don’t think so.

So the U.S. Department of Education, they talk about rigorous research, and in order to be strong evidence of what works, you have to have randomized control studies plus the quantity – there has to be two or more settings – and that gives you strong evidence. Well, I don’t know.

Can you ever find two classrooms that are varied that are similar on all the variables, on some of the variables? I don’t think so. Anyways, that’s my big beef about quantitative summary. It can be useful when testing well-established and consistent practice. Control is a really big challenge, and in times of rapid change, too early quantitative testing may mask beneficial positive capacity, in that oftentimes quantitative researchers really only can and are interested in one or two output variables, and you may find that you’re changing all of culture by the use of these connectivist pedagogies or tools, and it might be challenging to really pull them out using quantitative measures. And then the line reviewed random assignment studies – the gold standard – we just don’t spend enough money even to dream that it will ever happen in education.

OK, well maybe I’ll just pause in case anybody wants to grab the mic and have any comment at this point. Go ahead, Gary.

Gary: I want to make sure I understand exactly what you mean. Are you saying that some quantitative researchers are actually doing qualitative research but hiding it under the guise of more formal research on purpose or accidentally? I may be missing your point just a little bit.

Terry: OK, no I think quantitative researchers would say that they are trying to do the best quantitative research that they can do. Unfortunately, though, I think that when you get into naturalist environments, like real educational or real learning contexts, it’s almost impossible to do real quantitative research. So I’m not suggesting – in the sense that they’re controlling all the variables or explaining the variance that comes from all the variables that are impacting on the study. Maybe they’re violating some of the basic standards of what quantitative research demands. But are they deluding themselves? Well, I don’t know. Basically, and I will come to the point that I think we do far too little research in general, so I don’t mind having quantitative research being done, I think it’s all useful, but whether it’s really going to tell us what works or not, I just have serious doubts about it. Maybe my next slamming of qualitative research will help in a way.

So the qualitative paradigm, I have the man down on the bottom with the magnifying glass. They’re really trying to look at why things happen, all kinds of different modes, ethnographic – you know, qualitative, grounded theories, all kinds of different variations on a theme, but they’re basically trying to dig down to find why rather than just what or how much, and it presents special challenges in distance and distributed education because of the distance between participants and researchers. Oftentimes, you can’t do very good research or observations or ethnographic work because you don’t see the people in their real context; but it is the most common type of distance education research. At least it has been published, maybe because in some ways – well, whether it’s easier or not, it doesn’t have quite the entry level into a qualitative study I think is slightly lower.

Let me just give you an example. This was a recent study sponsored in the European Journal, Johnson (2007), “Dialogue and Construction of Knowledge in E-learning: Exploring Students’ Perceptions of Their Learning while using Blackboard’s Asynchronous Discussion Board.” You’ll notice that qualitative researchers like lots of words as exemplified by that title. These are the results that they found: there were four different ways of perceiving online learning: Rosemary thought it was practical experience; Sarah, Karen, Katherine, and Cindy thought it was about interconnections; Anthony and David thought it was about expressing one’s own thought; and Larry thought it was about flexible learning. And I think that’s all well and good, but didn’t we know that already, that people experience online learning for practical reasons, to express themselves, to make connections, and because it’s flexible?

So too many qualitative studies tell us what we already know. I could have picked out lots of others. I mean there are good qualitative studies, but generally you’ve got to make sure that it’s really an important question that you’re asking and you’re answering.

The measure of quality is critical appraisal concerning plausibility, internal consistency, and fit to prevailing wisdom. That’s according to Burkhardt and Schoenfeld that you can tell if it is a good qualitative study. But the problem comes from a practitioner’s point of view is that some of the qualitative studies that I’ve read, they start developing a whole new lingo that can really only be deciphered and read by other researchers. So if it’s a fit to prevailing wisdom, whose wisdom are we talking about? Are we talking about learners? Are we talking about teachers? Are we talking about researchers? It becomes a real challenge. Often the only answer that qualitative research gets to the trenches is “it depends.” Maybe I’m being too harsh, but there you go.

Let me finish my three straw dogs by talking about critical theory and critical research. This is again people talk about trying to find the hidden truth, or who is the power relationship, or where is the U.S. military hiding behind everything happening in my school – that type of question. Norm Friesen, a friend of mine now at Thompson Rivers, wrote a paper called “The Experience of Computer Use, Expert Knowledge, and User Know-How.” I got a couple of quotes that I pulled out of it. He’s showing the actual experience of computer use casts doubt in to the educational efficacy of computers understood as instruments of cognitive amplification or mind tools. I mean, to me, I guess maybe I go back to David Jonasson, but when he wrote the book on mind tools, I think that’s key to connectivism, the fact that tools are useful, and I don’t see how the actual experience can cast doubts into their efficiency, but anyways. Then his bottom one – user knowledge of the system appears as embodied, performative, and emphatically provisional. I mean, I don’t even know what that means, that sentence, so I don’t think it’s going to have too much effect on practitioners, but maybe – I’m sure there are people on this list – maybe Karen Swan can explain it to me.

So the problem with critical theory – there’s a place for it, all of us do it to some degree or another, it’s just that I’m too much of a pragmatist in trying to say that we’ve got lots to do, we can really improve education, and sitting around making a career out of finding all the holes out of it doesn’t strike me as being a positive contribution – but again, there’s my bias.

OK, so do any of these research paradigms meet the very real needs of practicing educators? Well, there was a really interesting study done in 1999 in Australia, where they got a bunch of teachers to read some published research, and they asked them, which of these studies do you find are really useful to you?

They found that the most pervasive, the most relevant, and most influential were the studies that addressed the relationship between teaching and learning – I think that’s positive – but there was no consistent result. Teachers are different.

But the conclusion of that study, I think, was most interesting – the findings from this study cast doubt on virtually every argument for the superiority of any particular research genre, whether the criteria for superiority is persuasiveness, relevance, or ability to influence practitioners. So there isn’t one perfect method; all of them have problems. But if you get people who come preaching at you that there is only one way to do research, they are probably wrong, and that we need to do a lot more research of all kinds.

And then researchers love to argue about which of their research methods is best, and I guess I’ve fallen into the trap already this morning, but a nice quote: “The dispute and the intensity of feeling is inversely proportionate to the value of the stakes” – and maybe that’s why academic politics are so bitter.

OK, let me now talk about design-based research. You may be less familiar with it, because it’s relatively new on the market or in educational research. The one thing I really like about design-based research is that it was developed and invented by educators for education. It isn’t imported from other fields directly, although it’s related to the whole idea of engineering and architectural research where they talk about design, where you design an intervention - something that you think, based on your knowledge, your reading, your understanding, your looking through the literature – that you’re going to design an intervention, so it’s intervention-based. Like an engineer or architect, you’re building something; you’re not going around just asking questions of everyone, you’re actually making a contribution, you’re doing something. So it focuses on the design, instruction, implementation, and adoption of some kind of initiative. It is related to development research, and it is also related directly to action research, and you’ll see that as I describe it further.

Two really interesting developments: the fact that context - I alluded to it earlier - and the attempts of quantitative researchers to eliminate it, what we’re finding is that the variability is pervasive and induced by the uniqueness of every single particular context. This resonates with constructivist learning, too, that the context of the individual has a tremendous impact.

So we can’t forget about context, and I like about design-based research that it doesn’t turn its back on context; it says context is happening, it’s real, it’s going to impact a study, and we have to be able to deal with that complexity. The next – the figural guy from Portugal writes quite a bit about context and he talks about how you have content but it has to match with context, and you can’t ignore one or the other.

So the other interesting thing, I think, is the whole talk, not new now, but the last 20 years or 25, chaos theory, complexity theory, and what we’re starting to realize is that educational environments are very complex environments, and it is not possible to precisely forecast or predict behavior. Hopefully we can explain it and we can help people function in these learning contexts, but we’re not likely to get a hundred percent predictability. Interventions, teacher interventions or whatever, are conditioned and recursively amplified or extinguished by contextual variables. So just because it worked in my classroom doesn’t mean it’s going to work in yours. In fact, emergent behavior happens. Students will appropriate technology in connectivist ways that we hadn’t even thought about, as will teachers, and we have to continuously support them in doing that, and we have to have a research environment that will support that innovation, that adaptation, that extinguishing of inappropriate behaviors instead of just running around trying to measure them or understand or control them.

So we have a design intervention operating in a context, and then we can’t forget about evaluation and assessment, because it drives formal education. If you can think of what would we be like if we didn’t have evaluation and assessment – a huge contextual variable as well to forget about, and I think design-based research has a way to handle it.

So what is design-based like? It’s iterative, it happens more than once; you don’t just try it once and forget about it, you try it in different contexts. It’s focused on the process as much as the output. It’s intervention, as I’ve said. It’s collaborative in multiple ways – collaborative between the researchers and the practitioners. It’s utility-orientated. It’s practical – comes based out of American pragmatism – James Dewey, all those guys who really thought philosophy matters, and if it matters, it really has to make an effect on practice. But unlike action research, it is much more likely to be theory-driven and generate theory. So it’s not just trying to do what will work only in my classroom; it’s trying to bring out the best of what others have discovered, as well, and designed.

Though you probably might have trouble reading this, but Brendan, Bain, and Whitland mapped what a design-based research study looks like and the four phases. The first is informed exploration, over here, where you look at the survey, the literature, you develop a theory, you talk to your audience, you do a needs analysis – you know, kind of common sense stuff. You use qualitative measures and you use quantitative measures. This is the kind of data you do. Often the problem with action research is that it is done by teachers who are real busy and they just don’t have time to do the informed exploration.

Sorry you can’t read this slide. Enactment is when you actually build something, change something, do something. You usually try to take account of the efficacy, how much it cost, what this intervention is all about, how much time it took. So you do quantitative gathering of that kind of data, you look at flowcharts, blah blah blah, and then you test it in some kind of evaluation in a local environment to which it was designed. Again, you use both qualitative and quantitative tools, you use them together - you videotape, you journal, all those sorts of things. And then finally, just because this thing has an impact in one environment doesn’t mean it’s going to have an impact on another environment, so you evaluate it in the broader impact: you talk about diffusion of innovation studies, you look at patterns, you try to paint the big picture from the small pictures put together. I’m sorry, I see there are some interesting comments going by, but I think I am going to try to hold them, because I’m not that good at multi-tasking; I’m older than twelve years old.

So to conclude, it’s a methodology designed by educators for educators. There’s been a number of – it’s a hot topic – special issues of these journals. I did a little bit of a bibliography of the articles written about design-based research: I put it up on the Cider site. And I wrote an article in the Canadian Journal of Learning Technologies – it won an award, actually – on design-based that you might be interested in.

So let me give you a very quick example of a design-based study that’s very much a work in progress, where we’re looking at social software interventions. I’m piloting them in my paced graduate-level course. I think their real application will come in continuous enrolment studies, but I’m having a little bit more trouble actually getting these interventions tested where I think they’ll be most effective, which is a bit of a challenge for me and for Athabasca University. The basic problem is what kinds of interactions needs are cost-effective and least restrictive on the freedom of both teachers and learners. A lot of people come to Athabasca University because it’s continuous enrolment. You can start a course any time, any day, or it’ll be rolled out to you when the post arrives or when you get a URL. So you don’t have to wait until September or January and you can finish it in two months or a year, depending on your time availability. So when you have that kind of environment, how do you get people to interact when not only do they not start at the same time, but they’ll be finishing at all sorts of different times. So it’s a special challenge and I think social software is an intervention that can make a difference.

Stage 1: Informed Exploration

So informed exploration, we reviewed the literature, we did interviews with course developers, we phoned people around the world who did continuous enrolment, we did a survey of students, and we published a paper on that, having your cake and eating it, too, and really what’s happening is we’ve got in distance education - we have independent study over here on the left, with correspondence, telecourse; we’ve got collaborative stuff like we’re doing right now; and what we’re trying to do is see if you can mesh these together to create a socially enhanced, independent learning-esque technology. Again, this is the sort of theoretical basis that we’re coming from, by taking stuff from the past, Hulsman’s stuff on the different types, and trying to make a coherent picture out of it, so that I can use that to try to talk people into participating in this research.

It’s really important learner freedom, learner space, pace, time, media, content, access, all that stuff - and Gary is brave enough to put his hand up, so go ahead, Gary.

Gary: Terry, can you go back to that last slide and elaborate a little bit more about the status of where that research is right now or where you are with that?

Terry: What I think, Gary, the best thing to do – I mean, this is just stuff I’ve written, more or less, and as I get through to the next stages of a design-based study, you’ll see where we’re at and what we’re doing right now, so maybe that will answer your question.

OK, so the problem with that paced, interactive, collaborative, third-generation type technologies is that they cramp the style of both teachers and learners, and is that value of pacing, of making you work together, is that really worth it and under what conditions and under what kind of students? So that’s the big problem that we’re trying to address.

So I think that educational social technology may allow us to do that and of course I had to jump out with my own definition of educational social network tools that support and encourage learning while retaining that individual control. So for me, and again from an Athabasca perspective, it’s quite important that that control remain with the students.

Again, I’ve already said it: what tools are needed to allow students in self-paced courses. OK. So we also did the survey.

We found that seventy-eight percent of our self-paced students indicate that they do want to interact, but they want to be able to work at their own pace. They want their cake and eat it, too. They want access to work of others. Now, you know, from Turn It In and plagiarism issues that might come as no surprise to any of you. But they prefer a synchronous or combinations, but only three percent preferred synchronous alone.

So you can see this informed exploration of a stage one: you do a lot of stuff in the literature, you do some exploratory stuff, you talk to people, you try to make sure that your intervention is going to be workable and meaningful.

Stage 2: Enactment

OK, the second stage is to interact, so we selected and installed an instance of the ELGG. We rechristened it “Me to You” – that should be dot athabascau.ca, not at (@) – typo there. We developed support strategies and documentation and we pilot-tested it. We also developed or listed some instructional design activities that can be used in any ELGG world and posted those. And we’re trying it in different courses and different classes. We’re trying it at the class level as well as to try to see whether people are interested at the program level.

Connectivist tools we used – we’re using wikis, blogs, event calendars, tasks, polls, user profiles, friends of friends, intersharing, group formation, connected with RSS, and trying to make connections happen both on and off line. So I see that somebody else, oh, 135 things – have to find out what’s out there. I’m now going to ask Nathalie to go ahead and grab the mic, Nathalie, if you have a question.

Nathalie: OK, I just wondered if you could expand on what you meant in the previous slide by “changing the instructional design of the courses, develop new instructional designs?”

Terry: What I meant by that is basically we look through the collaborative learning and literature and we try to find out what kind of instructional designs could work in a blogging, profile, syndicated kind of an environment. Donna Cameron and I wrote a list of things that are suggested activities, design learning activities, that teachers could try in this environment. They’re nothing much new, we just tried to isolate those to use the affordances of the ELGG tool set that we are using.

Image:Anderson-50.jpg

So this is the course that I’ve been teaching the last couple of years. Again, it is not continuous enrolment. I wish it was. I’m trying desperately to get some people to try it out and I’ve got some people nibbling around the edges at Athabasca, but it’s challenging to get change happening outside of my own little turf and so this is what I’ve been doing. I use Moodle for content and administration and for a little bit of the discussion, but I’ve been trying to wean people away from threaded discussions, and give them the exposure to blogging under the Me to You environment and the profiles that are available in the ELGG instance, using Elluminate for pacing, for social presence, real time, and using collaborative bookmarking, using FURL to try to get people to share their information. And the task, the construction they do, is they create learning objects or portals on topics that I pick and then they’re forced to create a learning object with the target audience being other master’s degree students, and so after three years we have ten or twelve of these portals that we’ve created. There’s the Me to You installation.

It is restricted to people at Athabasca University and that’s a contentious issue that we’re struggling with, whether security and privacy is important in order to post. With ELGG, it’s a nice tool in that the learners, the participants themselves, can decide whether their blog is available on the open Net or is restricted to people. People can read the unrestricted ones, but they can’t post without being associated with Athabasca, and we could talk about whether that’s a good idea or not later on.

Stage 3: Local Evaluation

So then we have local evaluation. We had to get ethics clearance, resolve some of the privacy issues. We’ve been doing interviews, focus groups with developers and faculty. We’ve been doing student satisfaction surveys with each of the different tools, interviewing students, looking at what tools they’re actually using, and then we’re trying to do a cost comparison on what it’s costing us to support these various tools. Multiple methodologies, as design-based research is, is agnostic, too; it isn’t morally opposed to either qualitative or quantitative study, but mixes them.

These are some of the results – you see a huge hand of nine – but, anyways, ask them which of these tools is most useful, and the Web conferencing, which is the Elluminate environment, seemed to be the most useful. And I was glad to see the blogs, though they were more confusing, had a much higher learning curve – because these were all guys who were used to threaded discussion – they did find some high value in them anyways. But which supported social and feeling connected? Again, the Web conferencing and the blogs discussion and e-mail came up high there as well. Mastering the knowledge objects – I’ll just spin through that stuff.

Of course, we talked qualitative data. “I saw blogs as additional labor and a place for personal commentary that didn’t add anything to my learning” – not too positive that one. But it wasn’t directly on task, and again, we’re getting peripheral learning happening and people blogging about it. They’re not just answering my questions or getting their ten percent for participation in the threaded discussion, but they are doing some interesting and useful stuff, at least by perception. So you see the differences in the individual variables that are happening there.

So we iterate those stages one to three multiple times, test different tools. We had to react to organization change, think about lifelong learning, staff development, decentralization in each of the iterations of the design-based study.

Stage 4: Broader Impact Evaluation and Theorizing

And then finally we get to stage four, where we start talking about the broader impact, we evaluate it, in theory used in paced courses, using different disciplines, try to collaborate with other ELGG groups, look at adoption of innovation – all the stuff that happens in the fourth level, which we’re not at yet at Athabasca.

So I hope that gives you an example of what a design-based research study looks like. It’s a lot of common sense, it’s pragmatic stuff – you do stuff, you do it well, you do it right, you do it in collaboration with actual practitioners, and you try to build understanding and knowledge that way.

So we need better tools, we need more funding and less fighting. And like this joke where Wally gets told he has to go and help Alice, so he asks her if she could maybe try working harder and then he’s worried that she’s going to get dependent upon him for that help – that’s what I don’t want to see happen. I think we have to work together, help each other as researchers and practitioners.

So a little plug for my own journal and how important it is to disseminate, get that stuff out there. I hope you’re subscribed to IRRODL, the price is right – free.

And CIDER is a portal for distance education research. I hope you’ve had a chance to check it out and you’re welcome to join and see the feature that it has.

And I’m not going to bother about that.

So educational research – grossly under-resourced to meet the magnitude of opportunity and demand. Paradigm wars are not really productive. And design-based research has a new model for us. And Web 2.0 is, I think, is critical to the kind of new knowledge growth, lifelong learning, that we’re all in need of and connectivism provides one of the important pedagogies for that.

OK, I’ll end with a little Confucius that you can read, and I’d like to open the door. I guess by my watch you’ve got seven minutes. I know I’ve blabbed a lot and I hope that some people have comments or questions.

George: By the way, once the microphone has been released, feel free to grab the mic, you don’t have to raise your hand once it’s been released, and feel free to ask some questions.

Kelly: Hi, Terry, Kelly Edmunds here. I really like what you’ve come up here, looking at the dynamics of a bunch of different tools, and how you’re working closely with those tools, looking at the design, the research, the impact. Can design research then go deeper? Can you just then isolate blogging and understand why that is good for socialization or for learning?

Terry: Yeah, definitely. I think the scope or the unit of analysis of the study scope is very flexible in design-based. I was doing an overboard kind of throw a lot of stuff at people, because it is a master’s of distance ed. that I’m involved in, so I don’t feel badly about exposing them to multiple tools, but you needn’t necessarily do that. I’m not suggesting at all – maybe I’m inducing a whole bunch of compounds by the different tools that I’m using all at the same time, having people getting confused the more so by all the passwords and all the other logistical issues that come into using multiple tools. You don’t have to. Maybe it would be better or easier to research if I tried to control those variables a bit, but design-based research does not say that you have to completely throw everyone into a brand new world. It’s talking about interventions, and those interventions can be very small or very large. So I’m glad you clarified that, Kelly, and welcome.

George: And I just wanted to ask a question that’s come up several times now in the discussion forum. I can officially say, Terry, that you have managed to get an awful lot of information into a short presentation, and I think we’ve had that essentially from every presenter in this conference so far, because there are so many things that we are dealing with here that you feel like you’re drinking from a fire hose, to use that analogy again. But excellent sequence, as Jason mentioned; it flowed very well. Several individuals have asked whether these PowerPoint slides would be made available. So I’ll throw that out to you whether you have objections to them being posted on the site. Elluminate will be made available. For those of you that are heading out here, I just want to chat about a few things as we’re heading out here as a means of wrapping up our conference. We’ll try and touch on that in a bit, but if you could just address that question, Terry.

Terry: No, I have no problem at all. In fact, various ones have been put up already, so they’re not all original to this conference, but I’d be glad to share them, so yeah, go ahead with them, George – including the two spelling errors I noticed.

George: Well, I did notice that you didn’t mention design and spelling explicitly as being reliant on each other, so I think that’s quite acceptable. I’ll turn the mic over to Martha.

Martha: Yes, thank you. Good morning, Terry. Thank you for your presentation. This is the co-chair from Calgary. I wanted to ask you one question that really worries me: Is the situation about funding research on education going to change or going to be more positive in the near future? What do you think? What are your perspectives as a well-known researcher in the area?

Terry: Well, I think it really depends on the politics and the region you live in. I’m very impressed with what’s happened, especially in the UK. They have their twenty, fifty centres of excellence on various topics and the EU funding for e-learning is huge compared to anything in North America. In Canada, I think we’re really struggling with the fact that we don’t have – we’re the only developed country in the world, anyway, that doesn’t have a national education policy and so we’re so defragmented and it’s very grim times in Canada. In the States, there’s a continuing amount for education research. I mean, I’m sure Karen and others here on this list could respond to that, but they are really under the gun of making that quantitative control group type studies as being the only acceptable kind, so it’s being channeled. I guess I’m not all that optimistic. I think we have to talk it up, we have to give credible research, we have to show that it can make a difference. There’s a sense that educational research doesn’t work, doesn’t help, and I find it hard to believe why we would think that. Maybe that’s the reason why we haven’t changed our educational paradigm for so long, but, I mean, we’re being pushed by lifelong learning and we’re being pushed by the accordances of the new technology, so I’m mildly hopeful, but not over optimistic – put it that way.

George: OK, well, I think there’s some excellent question here. There’s a few that are receiving the priored form as well. I’ve encouraged them, because there are certainly a lot of questions coming out of your presentation, Terry, and I think it links back to one of your earlier slides – the stakes are so high here that there’s a reason why the dialogue and the passion is there. So I’ve asked a few of those individuals who have mentioned or had some comments to please go into the Moodle forum, discuss it there.

I do also want to throw out a quick plug for a project we’re currently working on in our department here, the Learning Technologies Centre at the University of Manitoba. We’re currently in dialogue with establishing an open access journal that follows the Plus One with annotation following – little bit of some of the things Stephen and I were chatting about in the discussion forum. If anyone is interested in that, please let me know. We do have a group of individuals from Europe currently involved, a few individuals from Canada, and we do want to extend it to include American and Australian or Asian/Middle East representation. So if that fits into your boat, please let me know.

Secondly, I’ve also talked previously about Attender. I’d really love to see as many individuals as possible go into Attender and post their information there. It’s a nice way to see the global impact of the conference we’re currently having, so that’s fairly useful to look at.

The final point before I say my thank you to Terry: I would like to suggest that we do this debriefing of the conference thing. I don’t know the best way to do this. Perhaps we could do an informal session. I’ll propose this here: next week Monday, 11 o’clock, same time we started here. The title of the session would be “Next Steps.” You know, what are the practical next activities? If we get five people we’ll have a discussion; if we get a hundred, we’ll have another discussion. It’s just a way to put a bit of closure on this and have an opportunity to do some concrete things to move forward. That’s this next week Monday at 11 a.m., Central Standard Time. I’ll throw out an e-mail to the list, invite them to attend, and it’ll just be a discussion – what can we do? From there, Moodle will be open, so we do want to do some work with Moodle, and we’ll certainly tackle, play around with that, see, hold discussions in that space.

Final point: Terry, I think you’ve delivered an excellent presentation. You’ve given us a broad overview of some of the things we need to consider to move our industry forward. On the one hand, it looks dismal when we’re sitting at the .01 or .001 – I’m not even sure of the percentages – so that’s dismal, but I think that you provide us with a model to move forward. It’s one thing to be told of a problem; it’s quite another to be told of a problem and then presented with a potential solution. And I think it’s a solution that, much like what we’re doing with the Connectivism conference, it’s one that needs to co-evolve based on practitioner support and involvement in shaping the ideas, so it’s not a function that comes from the expert on a pedestal but from these lovely individuals in the trenches and co-creating the identity together.

So as the official close to the conference, thank you again for all your involvement, the individuals who have taken additional time to throw out different comments, to assist with setting up Second Life, getting the Moodle site cleaned up – thank you to Sylvia Curry for that, and others – it’s certainly been an eye-opening conference on my end. So I’d like to say, in conclusion, let’s give Terry a round of applause – excellent work on his presentation here today. I will hope to see you in this space again on Monday at 11, and I think that essentially wraps up the conference. I appreciate everyone’s involvement.

Resources

Design Based Research

Design Based Research Collective (DBRC): http://www.designbasedresearch.org/dbr.html

Journal of the Learning Sciences: http://www.cc.gatech.edu/lst/jls/

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141-178.

Collins, A. (1992). Towards a design science of education. In E. Scanlon & T. O'Shea (Eds.), New directions in educational technology (pp. 15-22). Berlin: Springer.

Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15-42.

DBRC. (January/February 2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8.

Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technolgy Research & Development, 53(4), 5-23.

Terry Anderson

Terry's home page: http://www.athabascau.ca/html/staff/academic/terrya.html

Terry's blog: http://terrya.edublogs.org/

The International Review of Research in Open and Distance Learning (IRRODL): http://www.irrodl.org/index.php/irrodl

Anderson,T. (2003). Getting the Mix Right Again: An Updated and Theoretical Rationale for Interaction. International Review of research on Open and distance Learning 4(2). Available at http://www.irrodl.org/content/v4.2/anderson.html (Equivalency Theorem)

Anderson,T. (2006). Interaction in learning and teaching on the Educational Semantic Web. In C. Juwah (Ed.), Interactions in online education: Implications for theory and practice (pp. 141-155). London & New York: Routledge. (ESW)

Theory and Practice of Online Learning (free e-Book) edited by Terry and Fathi Elloumi: http://cde.athabascau.ca/online_book/

Canadian Institute of Distance Education Research (CIDER) project: http://cider.athabascau.ca/

CIDER Design-Based Research SIG: http://cider.athabascau.ca/CIDERSIGs/DesignBasedSIG/

Athabasca University

Athabasca University (AU): http://www.athabascau.ca/

AU Center for Distance Education (CDE): http://cde.athabascau.ca/index.php

AU Online Graduate Programs: http://www.athabascau.ca/calendar/grad/

AU Online Undergraduate Programs: http://www.athabascau.ca/programs/outcomes.php

Canadian Association for Distance Education (CADE): http://www.cade-aced.ca/

Alberta Distance Education & Traning Association (ADETA): http://www.adeta.org/

Moodlemoot: http://moodlemoot.ca/moodle/index.php

Connection to Connectivism