All comments

Click this link to read

All 41 comments

Post-Theme 6: Qualitative and case study research

This last week we had two seminars for discussing Theme 6. 
On Monday the topic was Qualitative research and on Wednesday we discussed case study research.

Monday seminar

When discussing qualitative methodology, we talked about focus groups which I found interesting since I hadn't actually read any research paper that used this method.
I could relate to the type of focus groups I have participated in myself when attending a few different commercial marketing research sessions.
However the implementation of focus groups may vary between research projects, groups may be small or large and so on.

Focus groups as a research method can be used to gain understanding about a wide range of views in detail, and perhaps also get a collected
view when the group reaches consensus, by letting participants freely discuss a matter.
This makes it possible to obtain more answers than for example a strictly formed questionnaire could.

On the negative side, there's a risk that someone in the group takes over the discussion.

In order to gain some deeper understanding, I've read through this article Focus Group Methodology: Introduction and history


It includes a few examples, for instance one from a marketing research by the Coca-Cola Company,
on how a marketing disaster could have been prevented, had they only listened to the participants of the focus group in research performed prior to the launch of the new product.

It further suggests that focus group study is suitable for health and social science researchers. Media technology research is often on the subject social media and human-computer interaction and
is therefore closely connected with the above, making focus groups suitable for our field as well.

It brings up some further negative sides of focus groups:

• In some cases, the discussion may not go deep enough
• All participants may not actively take part in the discussion
• Some topics are not suitable to discuss in this way

We added the qualitative research paper that I had chosen "Perceived connections between information and communication technology use and mental symptoms among young adults-a qualitative study" to the course wiki.

Wednesday seminar

This is where I got a clearer picture what defines a case study, and I understand that the research paper which was basically a literature study is somewhat different to the empirical case study, still,
it was a very good match to the description by Eisenhardt in Building Theories from Case Study Research which I in preparation for this theme. It was great to get examples of different applications of the case study.

I realized that I had tried to explain the case study method only from the perspective of the literature study paper I had chosen, and to get some more general insight, I've also read through this (unpublished) paper on the subject from University of Texas by S. Soy.

She uses multiple sources to present the case study method and I think that this article could be a good resource for anyone who wants to engage in this type of research for their master thesis.
Even though I didn't actually learn many new facts from reading this after having read Eisenhardt and discussed it, it still confirms the conclusions we came to during the seminar. And it's naturally
always good to find multiple sources for information, something we should all understand by now.


So perhaps this short diplomatic summary of the case study method would be in place
They are complex because they are useful.

• Multiple sources of data
• Multiple cases
• Large amounts of data to analyze

Our group gave the first contribution to the description of what a case study is on the course wiki.
It must have been great since it's hardly been changed by the following groups?

Final word

It has been an interesting trip from Plato to modern research methodology and the knowledge I've gained from this course will certainly be put to use when it's time to do my master thesis. I want to thank everyone for the discussions around the many papers we've all read.

Pre-Theme 6: Qualitative and case study research

Qualitative study 
Perceived connections between information and communication technology use and mental symptoms among young adults - a qualitative study by Thomée et al. BMC Public Health, IF 2.08.

1. Which qualitative method or methods are used in the paper? Which are the benefits and limitations of using these methods?

Semi structured interviews, using open-ended questions, with 16 women and 16 men that had previously participated in a quantitative study on the same topic. Seems like a good way to get more details, and the quantitative data is already collected.

2. What did you learn about qualitative methods from reading the paper?

How data in a qualitative study is sorted out based on, in this case, by high ICT use and then categorized. Then these categories were compared and sorted and a model developed. The single quotes from individuals really tell a lot, however a problem is that it's a perception perhaps not always representative of the whole study group.

3. Which are the main methodological problems of the study? How could the use of the qualitative method or methods have been improved?

I found this to be an excellent qualitative study to read, and I could easily have been one of the participants. I recommend reading it! With my limited experience of this type of studies and from reading about possible weaknesses in qualitative studies, I must say that I could find none.

1. Briefly explain to a first year university student what a case study is.

According to what Eisenhardt writes in "Building Theories from Case Study Research", a case study is a research method that can include references to both quantitative and qualitative research. It can include quantitative research as means of evidence, and as protection against false conclusions from qualitative data. It normally also includes the richness of qualitative description to enable the construction of new theories and to explain the collected quantitative data.

A case study may include cases either randomly or orderly. The goal is to include samples of theories which can strengthen the new theory. During the research process, the case study may change data collection methods, since the goal is not to summarize observational data but to learn as much as possible from each case. In the paper this is referred to as "Controlled opportunism".

This type of research finishes when the theory cannot be further improved. Comparing the found theory to other literature can prove what novelty has been discovered. This novel theory is along with its' testability the strengths of a case study. Weaknesses of the resulting theory might include high complexity and idiosyncrasy.

The method may be useful when little is known about a phenomenon, when ideas around it are in conflict witch each other or with common sense.

1. Use the "Process of Building Theory from Case Study Research" (Eisenhardt, summarized in Table 1) to analyze the strengths and weaknesses of your selected paper.

Case study

Strategies for designing effective psychotherapeutic gaming interventions for children and adolescents by Goh et. al. from the journal Computers in Human Behavior, IF 2.067.
 -

Getting started

Mental health professionals would like to use computer games as treatment to mental illness in children. However, there's little evidence if the effectiveness of this method, explained with references to quantitative studies of mental health conditions in children.
 -

Method

The paper reviews computer gaming literature of "specified population" found by searching online databases. They have added articles and books of interested when "stumbled upon". A method described by Eisenhardt.

-

Goal

Create guidelines and strategies for psychotherapeutic game design.

It's explained that even though CBT has been successful, new methods should be explored, and children are comfortable using ICT.
-

Enfolding Literature

The paper compares contradicting studies of how computer games affect the mental state. Like described by Eisenhardt. It chooses to promote the positive theory, which indicates that games can be used as psychotherapeutic tools if correctly designed.

There's a discussion about strengths and weaknesses of games with the weaknesses being addiction and increased aggressiveness. All is backed up by references.
-

Analyzing Data

The authors bring up example studies on behaviour and computer games, most seem to be qualitative.

Ideas of good game design are presented with references. One problem found is that more boys than girls play computer games and there's no research that can point out the exact reason. I see a potential problem of the resulting theory due to this.

-

Reaching Closure / Conclusion

Games can't replace psychiatrists, but can be a good complement to traditional methods. 
The scope is narrowed towards the end by declaring that focus is not on successful treatment, but rather on acceptance of the game by the user.
-

The paper highlights important points for game development

• Reinforcement of behaviours
• Motivation, example:   

" Children in the classroom cannot always see the point of learning mathematics ... When they play computer games, they are usually working towards a clear objective..."

• Challenge
• Realism 

 Overall, good guidelines.

C'mon, give the kids a holodeck!

Further research will be needed to actually come up with what type of game to develop.

Some types of source studies have only been qualitative and cannot be generalized and they're lacking quantitative sources too. 

This "opportunistic" research method concludes that it relies on sources that do not exist yet, I suppose the positive side of that is that it's basically a suggestion to further research.

I found this to be a very interesting study. Computer games could be a very effective tools in the success of methods like CBT and for learning difficulties. Imagine if science reports were part of a cool game...

Post-Theme 5: Design research

Reflections on today's lectures for theme 5

First out was with Ylva Ferneaus with the topic Qualitative methods. 

We discussed the paper Ferneaus and Jacobsson. Questions were asked and conclusions were: Statistics could have made the paper more complete, but it was not a problem that there was no user study, it was not part of the scope of the paper. What made this project into research was that it was based on previous research and presented something new.

Ylva presented her own research philosophy which I found interesting:
Knowledge = Things taken notice of, caught attention
Research = Taking notice of something new + sharing it
Sharing knowledge = Getting others to take notice
 
I also found her description of a typical research process good:

1. You think everyone else has missed something
2. You check it up (literature, design explorations, reality)
3. You take notice (sometimes with surprise)
4. You convince them (text+images+diagrams+designs)
5. They accept it (publish, take notice, quote)

Even though I gave it critique in my last post, I realize now that this paper probably was a decent example of conceptual research.

Next was Haibo Li with an inspirational lecture on the topic Design Research.

The process of coming up with ideas

Firstly, it's important to filter ideas out, to find the best one(s). One might want to consider things like: 

• Does it address a real problem? 
• Appeal to market? 
• Is the timing right? 
• Will we be good at it? (or will we be eaten by the bear?)

Basically check the idea against a business plan. For non commercial research, these questions would be different.


Secondly, is the idea valid? Li gave an example of a bear chasing two people. Was the idea to outrun the bear? No, impossible. One person had to outrun the other to survive. Survival of the fittest. Typical business-thinking.

Evaluate the idea. Do some research. I suppose this is where we could apply conceptual research, in combination with quantitative or qualitative.

Communicate the idea
A good way to find a great idea is to think like a business-person and come up with statistics that can sell the idea.  Li also gave an example on how to convince us about an idea by laying out facts, and the point here was that we need to make use of mathematics.

Li's theory, which I also liked:
"If you want to become famous, solve the problem. If you want to be great, define the problem."
  
Problem solving
Li talked about how to define and solve a problem.

Here I might add some of my own work experience, not from research, but from troubleshooting products that were the result of research and development. Communication with people can be very difficult. In many cases, a problem might not be well defined, and you could find yourself pulling out your hair over a problem that is not correctly defined. Even for after-market service, small research projects can be necessary to actually find the root cause problem!



Li also said "Don't trust yourself." I agree and would add, don't trust anyone. Others will steal your ideas!

I also enjoyed some of the examples of great ideas, such as Vimio and Li's company Wawo.


Like I discovered and wrote about in Pre-Theme 4: Quantitative research, Li explained how Analysis of Variance (ANOVA) is a great method of analyzing and presenting statistics to convince people of a concept. It enables a scientific conclusion to be presented from charts, by comparing variability within groups and with variability in other groups.

Examples

1-way ANOVA: Does caffeine increase productivity of rats? 
2-way ANOVA: How does toy colours affect satisfaction in children, and depending on gender?


Both Ferneaus and Li brought up the problem of technology obsoletion, a critique I included in my Pre-Theme 5-post. 

Lastly, I stumbled across this article about Nobel prize winner Randy Schekman who wants to boycott the top science journals.
Schekman calls Impact Factor a "toxic influence". Oh well, I better start looking for two science papers for theme 6 with minimum IF of 1.

Pre-Theme 5: Design research

What the papers we've read for Theme 5 have in common

If Russell could have his saying, he'd probably say that they are sense-data based studies.

That is, they try to explore the way humans can interpret signals from electronic equipment.

Another more relevant term for this, Human-Computer-Interaction.

I see some fundamental problems with the usefulness of both these research papers.

The prototyping and ideas are based on currently available technology at that time.

These papers were published 2008-2009 and much has happened since then.

Knowing this, the research can still partially be useful for further studies.

Comics, Robots, Fashion and Programming:

outlining the concept of actDresses by Fernaeus and Jacobsson.

Reflect on the key points and what you learnt by reading the text.

The key idea was to use sign language to program or change functionality of electronic products by "dressing them up" with other items that symbolize a function.

The technology used for prototyping is rfid, is found in toys, price tags, key cards and road tolls.

It's a cheap and simple technology that can be easily integrated in almost any object, so it's very suitable for this concept.

As I indicated in the introduction, I find the concept obsolete.

I will make the assumption that most people that can afford a Roomba vacuum cleaner (I have one), also can afford a tablet or smartphone (I can).

The ActDress concept in its current form is suitable for toys and cheap products. But for more sophisticated items it seems impractical to have to dress things up. Since smartphones, computers or tablets normally can't read rfid, I'd make use of other available wireless technology such as WiFi or Bluetooth.

If further developed, ActDress could be an interesting technology for wireless mobile devices as symbols still can be used and presented on a screen as icons or graphical instructions. In fact, it's already here.

A visual programming language for my robot vacuum cleaner could easily be made available as an app for my tablet. I wouldn't need a special remote for the robot, there would be no need to add a display, or to dress it up.

FooFoo the Were-Rabbit costume for Roomba

Check the video: "Programming" that might be feasible for home appliances in a near future, we already have the technology in gaming consoles.

---

Turn Your Mobile Into the Ball: Rendering Live Football Game Using Vibration by Li et al 2008.

I think that this was an interesting conceptual proof even though I'm not very interested in football.

Today 2013, we can easily watch a full football game on a smartphone or tablet. However, the main idea with this paper was

to be able to get the information while the phone is in ones pocket. I'm not certain of why this would be a good idea.

Even though I find the concept mostly obsolete, it gave me an idea that could probably be implemented and commercialized.

1. How can media technologies be evaluated?

One good way is to present a prototype to a small test group (qualitative).

Or in some cases questionnaires like we saw in theme 4 could be suitable (quantitative).

2. What role will prototypes play in research? +

3. Why could it be necessary to develop a proof of concept prototype? +

5. How can design research be communicated/presented?

Prototypes serve an important role in research. They can be used as a proof-of-concept and to illustrate the theory.

I can give you some examples from real life. I've been working for a while in a company that develops, manufactures, sells and supports advanced machines.

These machines consist of hardware: Mechanics and electronics, and various software. Added to this, there is the concept of process knowledge (proper usage).

Whith a prototype, it is possible explain to and discuss the final product with other stakeholders. As a researcher or developer, you can get great feedback from other experts, sales, service and customers.

When I've been involved in such presentations/discussions, we usually sit down in a meeting room, look at a presentation or physical prototype, write comments/ideas on post-its and then the research either continues or it is decided that

the product is ready for development or the waste-bin. 
 

The process of evaluating a prototype at my job.

Research and development go hand in hand. After the concept has been approved, a development project can be started to realize the product.

4. What are characteristics and limitations of prototypes?

Prototypes often lack full functionality of the thought of final product. For instance, to demonstrate visual design, an object may be created that lacks the actual

function of the final product. The other way around, a prototype that demonstrates a function may lack appealing design and may not be made out of the same material or with the same methods

of manufacturing (can apply to both physical objects and code).

Normally during research and development, there is a need to manufacture several prototypes of varying design to evaluate which one will have the correct function, quality and cost of manufacturing.

Sometimes prototypes can actually be of better quality than the finished product, simply because of cost of production.

Post-Theme 4: Quantitative research

For this theme, I've read Bälter et al's paper which I (with my current experience) found to be an excellent example of a quantitative research design.
Then I selected and read another research paper, Information sharing on social media sites by Osatuyi, using a quantitative method as well. It was similar in the way that it used
a web questionnaire. Read all about it in the previous post.
After reading these I basically felt that I what I'd learnt was that I need to learn more about research methods and statistics.

During Monday's seminar with Hrastinski we discussed methods used in the selected papers.
Further, all groups made a flow-chart of the method used in the respective paper.
Our group chose Measuring Mobile Phone Use: Self-Report Versus Log Data by Boase and Ling, perhaps because two in the group had read it.
One amusing part was that they had interpreted it differently, which lead to an interesting discussion in the group.
Honestly I think that none of the flowcharts were fantastic. They were either unreadable or a bit confusing.
Still, flowcharts in general are very nice and useful when designing and presenting a process/method. Perhaps we all need to learn more about creating flow-charts for scientific research methods.

Today's seminar (Wednesday) with Olle Bälter was interesting. He arranged a competition style seminar to encourage us to find
advantages/disadvantages with quantitative and qualitative research methods. This was a fun arrangement, and I learnt about the topic even though my team finished last.
The only minor downside of this was that we didn't have time to take complete notes about the advantages/disadvantages.

On the topic of questionnaires, Bälter explained with some good examples what traps one might fall into while designing the questions.
I'll be sure to avoid ambiguous questions and negations when it's time for me to design one.
I understood that it's of great importance to evaluate the questionnaire before "release", or else the questions might be misinterpreted
and answered incorrectly. But what I found most interesting was the fact that successful questionnaires make use of our brains' need for novelty and dopamine reward, explained in another context (of why we tend to over-consume)
by Nate Hagens in the book Fleeing Vesuvius. The example Bälter brought up was a questionnaire that immediately after submitting the answers, presented a result of interesting facts like BMI.
It just makes it more fun and worth wile to answer questions if you get a result in the end. However, as I pointed out in the last post, sometimes rewards seem to have little effect. In the paper by Osatuyi, the students that were
supposed to answer could get extra points towards their grades if the answered, still only 57% answered, so perhaps there has to be a combination of rewards.
Finally, here's an interesting example of an upcoming reward system for answering questionnaires from Google:
Answer surveys to receive play store credit with Google opinion rewards app

Pre-Theme 4: Quantitative research

Information sharing on social media sites

B. Osatuyi, from the journal Computers in Human Behavior, Impact Factor: 2.067.

 

Problem

 Credibility with information shared on social media sites.

Research design - quantitative methods

Data was collected using a web based survey with e-mail invitations. It was of exploratory type. 200 university students were e-mailed a link to the survey. Only 114 students gave complete answers, even though all participating students were rewarded extra points for their grades.

Five social media technologies were included: 

1. Social networking
2. Micro-blogs
3. Wikis
4. Forums
5. Blogs

The definition of each information type explored in the paper is backed up by references to other studies.

The study used the following categories for information: 

• Personal (sensitive); health, relations
• Sensational; news, gossip, science
• Political; political discussions
• Casual; restaurant reviews etc

Chi-square test was used to analyze the results regarding which media technology is used to share information. The appropriateness of this method is explained by the ability of estimating if two variables are independent. The Chi-square test needs an estimated value to be compared with the observed value. I did not find any such estimations in this paper.

The study also introduces a classification code that the participants may use to indicate credibility of the shared information on social media. It's on a scale between 1 and 14 and contains fourteen combinations of the following credibility indicators: 

• Link to other sources
• Topic of interest
• Embedded video
• Embedded audio

However, it does not include the familiarity factor. My immediate thought is that familiarity with the person posting the information will affect the credibility quite a bit. I think they could have referenced such study. I found one research paper from University of Alabama called " Factors and effects of information credibility" that could've been relevant. However that type of research might be of more qualitative nature.

Above methods were used to reach the answer to their first question, "Does the codification of information credibility vary across different social media sites?"

The second to fifth questions were about if information shared varies on different social media. The variance of the answers (ANOVA) seem to have been analysed by calculating the degrees offreedom.

The analysis methods for these questions were the same. An example: the result of the first question in this series was that there was no statistical difference on which social media site people share personal information. I find myself at loss at criticizing the method since I lack the proper knowledge about it.

Conclusion and What did I find and learn?

The conclusion from this survey was that there is a difference between how people share information on social networking sites compared to other social media sites. It also finds that people are careful with what personal information they share. The researchers argue that organizations may use this to better engage their customers.

I think that they could have found a way to contact a larger initial group of potential participants since not many answer surveys. From personal experience, I will often not participate if not interested. Maybe this is a weakness of the method, only very motivated people will participate. 

Though they seem appropriate, for me it was a bit difficult to analyze the statistical methods due to limited knowledge of statistics. I had to look up the basics of ANOVA, Chi-squared test and degrees-of-freedom, but would still benefit from attending a statistics course.

---

Physical activity, stress, and self-reported upper respiratory tract infection  

Bälter et al.

Problem and Background

Many seek medical attention due to common cold and influenza (URTI).  The research focused on investigating the relation between URTI, physical activity and stress.

Previous studies of similar nature have been more qualitative and focused on smaller groups of athletes.

Research design

The study used a population-based prospective cohort method where about 1500 random participants were selected.

For the quantitative data collection, it used an adaptive web questionnaire, with E-mail reminders. It summarized the collected data from various questions, about health and lifestyle, automatically into points on MET-hour and Perceived Stress-scales, where MET is the rate of energy consumption.

The MET-hours were multiplied with reported hours spent on certain activities by questionnaire participants. There's a clear definition of how stress was measured with the PSS-scale.

Follow up questions on influenza-vaccine and allergies to rule out symptoms not related to URTI at the end of the data collection phase.

A clear diagram shows selection and filtering of random participants in for study. Nice tables are presented of the collected multi-variable data. Incomplete results were not used.

Discussion and Conclusion

The conclusion from the analysis was that physical activity lowers the risk of URTI for both sexes. It also states that highly stressed men benefit the most from physical activity. The study could confirm findings from other similar research regarding the relation between physical activity and URTI, except for one study example where the test group consisted of professional athletes. It could however not clearly confirm an overall relation between stress and URTI despite attempts to exclude/select parts of the data to make it fit previous findings. Though there was apparently a stronger relation between stress and URTI in men, perhaps because the way men react to stressful situations.

I think that they satisfactory discuss weak points in the data and explain why it probably does not affect the overall result. The conclusions can be useful in further studies, and as a resource in informing the public about exercise, illness and stress. I think that this was an excellent report and the scope is wide enough, I wouldn't change anything really.

---

Which are the benefits and limitations of using quantitative methods?

From reading the research by Bälter, it seems that it's easier to get a result that represents the majority if a large and random group is studied. If the subject group is too small, the result may not be representative.  Also, occurrence of  phenomena may be missed out due to the focus on testing a hypothesis rather than producing one.

Which are the benefits and limitations of using qualitative methods?

Since the qualitative study selects a smaller group to study,  it can give a clear and detailed description on an individual and specific level. It can, as opposed to a quantitative study, react and change focus based on findings during the study. However, collection of data might be more difficult and takes more time, the results might not be applicable to other groups, and the result could be influenced by the researchers subjectivity.

References
University of South Alabama 2013,

Strengths and Weaknesses of Quantitative Research
Strengths and Weaknesses of Qualitative Research