Summary of Research
This is based on the following published research:
| Citation | Webb, R., Watson, D., Shepherd, C. and Cook, S. (2021). Flipping the Classroom: Is it the type of flipping that adds value?, Studies in Higher Education, 46(8): 1649-1663. |
| Abstract | Interest in the flipped classroom in higher education has burgeoned despite the literature revealing that the effects on assessment outcomes remain elusive. In this paper, we present the results of an empirical analysis designed to compare the impact on assessment outcomes of different approaches to the flipped classroom (didactic and non-didactic). Focusing on a cohort of Intermediate Economics students we investigated the influence of these approaches on their examination results by utilising an OLS (Ordinary Least Squares) regression and probit followed by quantile regression. Our analysis revealed small positive effects when students were exposed to the ‘non-didactic’ flipped classroom but no effect when pre-lecture materials were used didactically to mimic the material given in traditional lectures. This study demonstrates the need for further meta-analysis and longitudinal studies to investigate the relationship between different forms of the flipped classroom and student assessment outcomes. |
| DOI | https://doi.org/10.1080/03075079.2019.1698535 |
Takeaway Message: In relation to the impact of flipping on student examination performance, our analysis indicates assessment outcome gains from flipping the classroom cannot be taken for granted and that the design of pre-class and in-class materials is critical. In particular, if the ‘flipped classroom’ is constructed around ‘lecture capture’ based on past/older material, and the approach is didactic in the treatment of the subject area, our study suggests a positive impact on student assessment outcomes is less likely to be observed. By mimicking the standard lecture approach, the satisficing student will simply use the additional resources as a time saving opportunity. To derive positive effects, pedagogy must be adopted with more precision. Our ‘non-didactic flipped classroom’, for example, generated small positive effects on examination performance of nearly 4%.
1. Background
“Some people talk in their sleep. Lecturers talk while other people sleep.” Camus
This epigram is a mainstay of those ridiculing the value of the lecture format in delivering learning outcomes associated with higher education. With Bligh (2000) typically used as the backbone for this anti-lecture stance, the rhetoric used tends to be rather combative in tone. Examples include “the lecture is dead” (Folley, 2010) and “drone warfare” (Gibbs, 2013). In support of this pugnaciousness, there is seemingly a wealth of sources in support. Gibbs (2013), for example, refers to how “more than 700 studies have confirmed that lectures are less effective than a wide range of methods for achieving almost every educational goal you can think of”. Undoubtedly, we can refer to limitations that the lecture format is more susceptible to. Schmidt et al. (2015), for example, focuses on how lectures encourage a ‘superficial and abbreviated’ outlook inconsistent with the promotion of critical thinking. In addition, time constraints can lead to an overly regimented approach which destroys any possibility to cater for the diverse variation in the students’ skills set. McFarlin (2008), for example, identifies how students prefer self-pacing in learning. Students may subsequently vote with their feet, with low lecture attendance rates becoming the norm. However, despite these issues, the available pedagogical literature should be treated with caution. Providing a definition for what a lecture entails is seemingly straight-forward. Bligh (2000) writes that lectures “are more or less continuous expositions by a speaker who wants the audience to learn something”. In reality, however, there is no clear-cut definition which enables support for an anti-lecture consensus. From the charisma of the lecturer to the diversity in pedagogical approaches that are adopted, there is such disparity in approach that any unambiguous conclusion of the relative failure of the lecture format is not feasible. It is therefore perhaps unsurprising that general rules of thumb, lacking any robust empirical background, are often at the forefront of comment. This is demonstrated by the so-called ‘pyramid of learning’ which categorises learning gains very precisely, but without any robust empirical support: we supposedly remember 5% of information from lectures; 10% from what we read; 20% presented from audio-visual mechanisms; 30% from demonstrations and 50% through participation in discussions.
Despite the remaining question marks over the effectiveness of the lecture format, the latest pedagogical fashion- ‘the flipped classroom’ as pioneered by the likes of Bergmann and Sams (2012)- is portrayed as the panacea to all of the lecture’s woes. The perceived solution is simple: students are asked to prepare prior to the lecture, enabling the lecturer to change his/her approach such that the student is placed at the centre of the learning experience. The practical reality of this approach is diverse. However, a common approach is to record the standard lecture content beforehand. These videos can then be uploaded to a virtual learning environment and, in conjunction with audience response systems, the actual lecture can instead be used for more interactive exercises. This provides an opportunity for self-pacing of the didactic transmission of key knowledge. It enables greater discussion between student and lecturer, allowing identification of what students found clear and where they possibly require further help. Berrett (2012), in support of this premise, states that “the immediacy of teaching in this way enables students’ misconceptions to be corrected well before they emerge on a midterm or final exam”.
Promoting self-confidence and going beyond superficial transmission of lecture slides, the possible gains do seem decisive. The empirical evidence, however, is rather mixed. Bishop and Verleger’s (2013) review points to the lack of anything concrete in support of the noble flipping aims. There are also numerous examples that suggest that, while there may be improvements in student satisfaction, there is no significant impact on assessment performance (e.g. Blair et al., 2015; Guerrero et al., 2015; Sparks, 2013). However, in economics education the research overall does tend to generate a more encouraging outlook (e.g. Olitsky and Cosgove, 2016; Caviglia-Harris, 2016). The debate here seems to be more focused on the size of any effect, with some suggestion that it is relatively modest and therefore pedagogical gains should not be exaggerated.
This paper adds to the debate on flipping, and the nature of the lecture format in general, by considering an expanded view of the inverted classroom:
“an educational technique that consists of two parts: interactive group learning activities inside the classroom, and direct computer-based individual instruction outside the classroom.” Bishop and Verleger (2013)
It will argue that it is the nature of the computer-based instruction outside the classroom which is the key component. Thus we will argue that positive effects from flipping are less likely when these pre-lecture materials are used to merely mimic the material given in ‘traditional’ lectures. As shown in the following discussion, this viewpoint is derived by integrating a behavioural analysis describing the ‘satisficing’ student with the impact of introducing pluralist teaching approaches on learning. To ensure we go beyond opinion, we test our approach through an empirical analysis into assessment outcomes for a 2015-16 student cohort of Intermediate Economics students. Overall, we argue that the integral importance of a module’s pedagogical design will naturally lead to a divergent literature into estimated pedagogical benefits. As with the perceived value of the lecture format, there will not be any automatic gains from the use of the flipped classroom. Rather than suggesting this approach should not be used, it informs us that these debates cannot be treated in isolation with the general concern that economics is often taught in a one dimensional way which focuses too much on neoclassical orthodox.
2. The Innovation Puzzle
Technology undoubtedly has enabled radical change in our teaching provision. This is ably illustrated through the rapid rise of ‘blended learning’ methods. Described as combining traditional classroom activities with the virtual environment and web-based technologies, the President of Pennsylvania State University described such methods as “the single greatest unrecognized trend in higher education today” (Young, 2002, p. 33). Evidence of improved learning outcomes, however, provides a decidedly mixed picture. There is some evidence of positive gains. Chen and Lin (2012), for example, refer to how supplemental video lectures generate an overall improvement of 4% points on exam outcomes. Kwak et al. (2015), in contrast, report strong negative effects when learning is deemed to be cumulative. However, arguably the most likely outcome are statistically insignificant effects in comparison with traditional teaching methods (Brown and Liedholm, 2002; Terry and Lewer, 2003; Olitsky and Cosgrove, 2013). It is the failure to trace a clear link between innovation and assessment outcome which requires an explanation.
A possible answer is offered by Allgood (2001) and the notion of ‘grade target’ student behaviour. This builds on the framework by Harackiewicz et al. (1997) which notes that the student’s motivation is dependent on two goals: a mastery goal describing “the desire to develop competency” (i.e. to learn as much as possible); and a performance goal “to evaluate performance relative to some benchmark” (i.e. to complete work whilst effort minimising). The relative importance of these goals is likely to be dependent on the student’s rationale for attending university:
“…the primary reason many students enter college is to get a job with a desired set of characteristics. Students who view their education in this manner are likely to be motivated to set goals that allow them to reach a desired level of performance with minimum effort” Allgood (2001, p.486)
Evidence over student motivation is offered in the following table, further supporting Allgood’s case.
Table 1: Student motivations to attend British universities
| Type | Main Purpose | Percentage |
| Next Steppers | To assist in clear career goals | 60% |
| Toe Dippers | To benefit from the student lifestyle/living experience | 9% |
| Option Openers | To learn about interesting subjects | 19% |
| Academics | To be stretched intellectually | 22% |
Source: National Union of Students (2008)
The majority of students are shown to be career motivated. Although the ‘option openers’ may share some traits, the small percentage of ‘academics’ suggests that mastery goals are unlikely to be as substantive as performance goals.
The available theoretical and empirical evidence therefore points to a grade target perspective. This means that, with any teaching innovation, we have to be mindful that any positive impact may be skewed in favour of student feedback. By lowering the time costs associated with acquiring the required knowledge, the innovation may simply enable more time to pursue alternative non-academic activities. It is perhaps worthwhile to note that finding evidence of these positive effects on student utility is straightforward. Overbaugh and Nickel (2011, p. 165), for example, write the following:
“When a blended course is developed, supported and implemented well, researchers have found that a majority of the students will be as satisfied or more satisfied with the blended course as they have with previous face-to-face courses.”
To show that ‘the flipped classroom’ goes beyond these effects we cannot restrict our analysis to student satisfaction data. Instead, an empirical approach is required which links the student’s motivation to learn and their assessment outcome. The next section describes our attempt at meeting this requirement, where we acknowledge that providing for different styles of pre-lecture materials will enable a more in-depth investigation that goes beyond a simple binary ‘flipping versus traditional’ outlook.
3. Construction of Pre-Lecture Resources
A familiar flipping starting-point is to use recorded lectures which have been given to a previous cohort. This is often adopted out of convenience, given video capture software has become an increasingly common tool across higher education. The Campus Computing Project (2014), for example, reports that 9.4% of classes in US public universities use such software. Our approach investigates whether providing these traditional resources may actually be counter-productive, given the motivation to learn is left unchanged. This involves the creation of two distinct pre-lecture materials: ‘Monist Pre-Lecture’ versus ‘Pluralist Pre-Lecture’.
The ‘Monist Pre-Lecture’ resources effectively replicate the standard lecture capture format. Using a variety of screen-casting and multimedia software such as ‘Videoscribe’, complimentary materials for a standard intermediate microeconomics textbook are uploaded weekly onto the student’s virtual learning environment. The term ‘monist’ is adopted as the treatment of microeconomics is restricted to the standard neoclassical framework adopted in intermediate microeconomics. This means that alternative schools of thought are not initially mentioned. By providing a clear narrative, the resources are used to fully describe the technical demands of the subject material. This is highlighted through the application of numerous problem-based questions, demonstrating the core mathematical nature of intermediate microeconomics. Rather than changing curriculum, the approach adopted focuses on the so-called ‘multimedia principle’. This asserts that the student achieves greater deeper learning than from text alone, as illustrated by Mayer (2014, p.43):
“The cognitive theory of multimedia learning is based on three cognitive science principles of learning: the human information processing system includes dual channels for visual/pictorial and auditory/verbal processing, each channel has a limited capacity for processing, and active learning entails carrying out a coordinated set of cognitive processes during learning.”
The ‘flipped classroom’ can then provide for an interactive lecture environment that subsequently engineers two key learning gains. First, it provides opportunities for further discussion of any of the material that students have found confusing. Second, and arguably integral to the pedagogical approach, there can be a focus on any perceived theoretical limitations in the material that has been covered in the pre-lecture resources. The ‘flipped classroom’ then supposedly enables the delivery of both technical analysis and the critical outlook required in essays.
Our second resource is entitled ‘Pluralist Pre-Lecture’. Here, e-learning materials are constructed, from the outset, to recognise that no single microeconomic theory can be treated as complete and infallible. Instead, the resources are deliberately assembled to compare and contrast between multiple schools of economic thought which “co-exist but also interact” (Mearman, 2010). Rather than just generating an orthodox-heterodox theoretical comparison, aspects of the topic are separated according to whether particular approaches can, or cannot, assist our understanding. The schools of thought included depends on the individual topic areas. However, generally speaking, the focus is on neoclassical, Institutionalism, Marxist, Austrian Economics, Ecological and Feminist Economics. Unlike the ‘Monist Pre-Lecture’, there is no clear narrative and therefore more student effort is required to appreciate the purposes of the analysis provided. Rather than simply trying to replace the neoclassical orthodoxy with an alternative, the materials are designed to show how specific economic outcomes can be explained by alternative perspectives. No clear conclusions are provided, with the student left to consider which explanations they find more coherent and more realistic.
The ‘flipped classroom’ again engineers further two key learning gains. First, given the increase in information generated by comparison of schools of thought, further detail can be provided in the lecture. For example, empirical research can be added to further compliment the debate over the relevance of both complimentary and competing theoretical perspectives. Second, audience response can be used to survey student perspectives and play ‘devil’s advocate’ games where the lecture focuses on critiquing those perspectives.
A flaw in our approach is that, as discrimination across students over the availability of resources is not possible, we are not able to randomly allocate students between our two types of pre-lecture material. To minimise any empirical problems, we therefore adopt the following procedure. First, to ensure that we can adequately test for the differential effects of learning resource, the lectures topics are split equally and randomly assigned across the two forms of pre-lecture materials. This ensures that they are equally relevant as revision aids for the final examination. Second, the general structure of the pre-lecture materials is communicated to students. Given it is unlikely that all students will comply and watch every pre-lecture resource, student preferences are likely to lead to pre-selection according to preference.
We hypothesise that the ‘Monist Pre-Lecture’ material will simply make it more straight-forward for the target-orientated student to meet their objective. This would suggest that these methods are less likely to positively impact on student outcomes. In contrast, the ‘Pluralist Pre-Lecture’ is driven by a commitment to encouraging debate from the onset. Perceptions of a simple truth are avoided and, from the beginning of their learning of the topic, students are encouraged to maintain an open mind. This hypothetically can change their motivation for attending the lecture. Forming their own opinions over the conflicting approaches, lectures are already geared to engaging debate and the instructor becomes a guide in that process.
4. Empirical Findings
Our empirical approach follows the work by Moffatt and Robinson (2015). This uses quantile regression to provide a more complete understanding of the impact of teaching innovation on the distribution of student performance. While Ordinary Least Squares (OLS) provides estimates of the mean effect on performance, quantile regression allows us to differentiate between effects on relatively low performers and relatively high performers. Using this approach, Moffatt and Robinson (2015) dismiss any significant positive effect of online multiple choice revision quizzes within the Virtual Learning Environment. As our online resources also include quizzes, we therefore ultimately also provide a check for this paper’s findings.
Before the quantile approach is used, we first provide OLS and probit findings. More formally, the following specifications are adopted:
where:
Exami is the mark received in the final intermediate microeconomics exam
GoodHi is a dummy that equals 1 if a ‘good honours’ outcome is achieved (i.e. 60% +)
Ability is proxied by two variables: exam outcome in Introductory Microeconomics as students build on their knowledge and mark achieved in a discussion board for critical evaluation skills.
XMonist is the number of ‘Monist Pre-Lectures’ resources that have been read by the student.
XPlural is the number of ‘Pluralist Pre-Lectures’ resources that have been read by the student.
XQuiz is the number of quizzes completed by the student.
Gender is a dummy that equals 1 if the student is male.
Overseas is a dummy that equals 1 if the student is an overseas student.
JointH is a dummy that equals 1 if the student is on a joint honours scheme.
From table 1 we can see that there are two key characteristics of our e-learning data. First, the ‘Monist Pre-Lecture’ format is slightly more popular than the ‘Pluralist Pre-Lecture’ alternative. Second, the overall student commitment to the ‘flipped classroom’ framework is disappointing. On average, less than half of the material is read. Moreover, a very low percentage of the quizzes are completed.
Table 1: Raw Data Descriptions
| Mean | St. Dev | Min | Max | |
| Exam (%) | 62.1 | 17.2 | 10.0 | 94.0 |
| Good Honours (60%+=1) | 0.633 | 0.484 | 0 | 1.0 |
| Ability 1 (Introductory Economics mark) | 59.5 | 16.85 | 12 | 89 |
| Ability 2 (Discussion Board mark) | 63.0 | 19.2 | 0 | 93.0 |
| % Monist read | 47.92 | 33.14 | 0 | 100.0 |
| % Pluralist read | 41.45 | 35.43 | 0 | 100.0 |
| % Quizzes finished | 4.79 | 12.03 | 0 | 62.5 |
| Gender (male=1) | 0.7 | 0.460 | 0 | 1.0 |
| Overseas (=1) | 0.35 | 0.479 | 0 | 1.0 |
| Joint Honours (=1) | 0.25 | 0.435 | 0 | 1.0 |
| Number of Observations | 120 |
Given we are unable to separate our cohort such that they only have access to one form of pre-lecture material, our results may suffer from a close correlation between reading of the pluralist and monist resources. We could simply have a measure of engagement, with the engaged utilising both forms and the non-engaged ignoring both types. The partial correlation, however, is found to be 0.579. In addition, in none of the multiple specifications investigated by the authors is the coefficient on XMonist found to be statistically specific. This includes simple specification which excludes both XPlural and XQuiz variables. Table 2 provides an example of our findings, utilising all of our available variables.
Table 2: OLS and Probit Results
| OLS (dependent: exam mark) | Probit (dependent: good honours) | |
| Ability 1 | 0.169** (0.075) | 0.019**(0.009) |
| Ability 2 | 0.273***(0.077) | 0.002 (0.008) |
| XMonist | -1.068 (1.202) | -0.118 (0.133) |
| XPlural | 3.557***(1.123) | 0.451***(0.136) |
| XQuiz | 0.544 (1.434) | 0.182 (0.220) |
| Gender | -1.685 (3.024) | -0.177 (0.337) |
| Overseas | -5.907*(3.190) | -0.676*(0.346) |
| Joint Honours | -8.691***(2.955) | -0.769**(0.309) |
| Constant | 36.236***(7.013) | -0.843 (0.756) |
| R-squared Measure | 0.403 | 0.280 |
Standard errors in parentheses. *** p<0.01, **p<0.05, *p<0.1
The OLS results are broadly consistent with Moffatt and Robinson (2015). Our control variables are arguably as expected. Previous exam results, proxying differences in overall microeconomic knowledge, is positive and statistically significant. The discussion board outcome, controlling for differences in critical evaluation skills, is similarly positive and significant. There are also significant differences between students on single and joint honours, perhaps reflecting difficulties created by any reduced exposure to general microeconomic tools. The significant differences between home and overseas students is also notable. Given exam questions tend to be more essay-orientated it highlights the general need for continuing investment in the provision of generic skills. As with Moffatt and Robinson (2015), we find no evidence that completing online quizzes positively affects student performance levels.
The key conclusion, however, is that there is significant difference between the impact of our monist and pluralist pre-lecture materials. There is no evidence that ‘Monist Pre-Lecture’ resources positively impact on student performance levels. In contrast, ‘Pluralist Pre-Lecture’ resources have a significantly positive effect and suggest an increase of over 4 marks on student outcomes. This is in line with our premise that, without a significant change in the motivation to learn, these pedagogical innovations are likely to have no impact on student outcomes. These results are also supported by the probit analysis into good honours outcomes.
Figure 1: Coefficient estimates from quantile regression
Notes: Quantiles are on the horizontal axis. Shaded area provides a 95% confidence interval for the quantile regression estimates. The solid straight line represents the OLS variable coefficient (with the dashed lines indicating the confidence interval)
To further investigate these effects, we now turn to the quantile regression approach. Overall there is no substantial evidence that this approach is more revealing than standard OLS. For brevity, as shown in Figure 1, we focus on the coefficient estimates on our pre-lecture resources. The results for ‘Monist Pre-Lecture’ broadly confirm our previous results. However, at least for some lower quantiles, there is evidence of a small negative effect. Understanding this loss in performance is difficult, but confirms that using previous lecture materials for the ‘flipped classroom’ environment may be counter-productive. Our ‘Pluralist Pre-Lecture’ outcome indicates that the effects are smaller for the higher quantiles. Going back to the National Union of Students (2008) breakdown into student motivations, the higher quantiles are likely to include the students looking to be stretched intellectually. It is therefore probable that the change in resources will impact on the academic motivated students less, given they are typically already committed to reading around the subject area. The higher effects on the lower quantiles is therefore consistent with changing motivation to learn on the ‘grade targeting’ student. There also could be positive effects from higher engagement, as the technical material that is traditionally covered in intermediate economics can be understood better as it is embedded within wider economic debates.
5. Conclusions
Empirical analysis into pedagogical innovations is not a straight-forward proposition. We would naturally expect a wide range of outcomes, reflecting the impact of subtle differences in instructor techniques. This paper has added to that debate through the consideration of ‘flipped classroom’ techniques. By constructing two different forms of pre-lecture materials for student viewing, it highlights how assessment outcome gains are not guaranteed. If the ‘flipped classroom’ is constructed around ‘lecture capture’ in order to enable greater interactivity within the lecture, positive assessment outcomes are less likely to be observed. By simply mimicking the standard lecture approach adopted in economics, we can assume that the cost minimising student will simply use the additional resources as a time saver opportunity.
This paper does not argue that ‘flipping the classroom’ should be rejected. However, the conclusions that we derive are still strong. ‘Flipping’, rather than representing a radical shift in teaching methods in economics, can be deemed to be subservient to the pluralist agenda. We do find some evidence that, if it is used to change the motivation to learn and further enhance student engagement, it can have significantly positive effects. However, this requires an instructor who is generally interested in radically transforming the curriculum covered in economics education. Without that radical motivation, ‘flipping’ can be criticised as the current fad which, if anything, is associated with unjustifiable high set-up costs.
References
Allgood S. (2001). Grade Targets and Teaching Innovations, Economics of Education Review, 20(5): pp. 4835-403.
Bergmann, J. and Sams, A. (2012). Flip Your Classroom: Reach Every Student in Every Class Every Day. ISTE.
Berrett, D. (2012). How ‘Flipping’ the Classroom Can Improve the Traditional Lecture, The Chronicle of Higher Education, p. 12.
Bishop, J. L. and Vergeler, M. A. (2013). The flipped classroom: A survey of the research. Paper presented at the 120th ASEE Annual Conference & Exposition (Paper ID #6219). Retrieved August 27, 2013, from http://www.asee.org/public/conferences/20/ papers/6219/view
Blair E., Maharaj C., Primus S. (2015) Performance and perception in the flipped classroom. Education and Information Technologies.
Bligh, D. (2000) What’s the Use of Lectures? San Francisco: Jossey-Bass.
Brown, B.W. and Liedholm, C.E. (2002). Can web courses replace the classroom in Principles of Microeconomics? American Economic Review Papers and Proceedings, Vol. 92(2), pp. 444-8.
Camus A. ‘‘Some people talk in their sleep. Lecturers talk while other people sleep.’’ as an epigram at: Long A., Lock B. 2008. Lectures and large groups, ASME, Lecturing & Listening, Understanding Medical Education Series, pp. 1–28.
Chen, J. and Lin, T.F. (2012). Do supplemental online recorded lectures help students learn microeconomics? International Review of Economics Education, Vol. 11(1), pp. 6–15.
Folley D. (2010). The Lecture is Dead Long Live the e-Lecture. Electronic Journal of e-Learning, Vol. 8 Issue 2, pp. 93-100.
Gibbs G. (2013), Lectures don’t work, but we keep using them. Can a demonstrably ineffective pedagogic form still be put to good use? Time Higher Education.
Guerrero S., Beal M., Lamb C., Sonderegger D. and Baumgartel D. (2015). Flipping Undergraduate Finite Mathematics: Findings and Implications, PRIMUS: Problems, Resources, and Issues in Mathematics Undergraduate Studies, 25(9-10): 814-832
Harackiewicz, J.M., Barron, K.E., Carter, S.M., Lehto, A.T. and Elliot, A.J. (1997). Predictors and consequences of achievement goals in the college classroom: Maintaining interest and making the grade, Journal of Personality and Social Psychology, 73(6): 1284-1295.
Kwak, D.K., Menezes, F.M. and Sherwood, C. (2015). Assessing the impact of blended learning on student performance. Economic Record, Vol. 91(292), pp. 91-106.
Lancaster, S.J. (2013) The flipped lecture. New Directions 9(1): 28-32.
Mayer, R.E. (2014). The Cambridge Handbook of Multimedia Learning (Second Edition), Cambridge University Press, New York.
McFarlin, B. K. (2008). Hybrid Lecture-online Format Increases Student Grades in an Undergraduate Exercise Physiology Course at a Large Urban University, Advances in Physiology Education, 32 (1): 86-91.
Mearman, A. (2010). Pluralism, heterodoxy and the rhetoric of distinction, Review of Radical Political Economics. 43(4): 552-561
Moffatt, J.M. and Robinson, C. (2015). Virtual Learning Environments: Linking participation to evaluation, International Review of Economics Education, 19: 22-35.
National Union of Students (2008). NUS Student Experience Report. Available at: http://www.nus.org.uk/PageFiles/4017/NUS_StudentExperienceReport.pdf.
Olitsky, N.H. and Cosgrove, S.B. (2013). The effect of blended courses on student learning: Evidence from introductory economics courses. International Review of Economics Education, Vol. 15, pp. 17–31.
Olitsky, N.H. and Cosgrove, S.B. (2016). The better blend? Flipping the principles of microeconomics classroom, 16: pp. 1-11.
Overbaugh, R.C. and Nickel, C.E. (2011). A comparison of student satisfaction and value of academic community between blended and online sections of a university-level educational foundations course, The Internet and Higher Education, 14(3): pp. 164-174.
Schmidt H.G., Wagener S.L. , Smeets G., Keemink L.M. and van der Molen, H.T. (2015), On the Use and Misuse of Lectures in Higher Education, Health Professions Education, Vol. 1, Issue 1, pp. 12–18.
Sparks, R.J. (2013). Flipping the Classroom: An Empirical Study Examining Student Learning, Journal of Learning in Higher Education, 9(2): 65-70.
Terry, N. and Lewer, J. (2003). Campus, online, or hybrid: an assessment of instruction modes. Journal of Economics and Economic Education Review, Vol. 4(1), pp. 23–34.
The Campus Computing Project (2014). Campus Computing Survey. Available at: <http://www.campuscomputing.net/> [Accessed 25 July 2016].
Young, J. (2002). “Hybrid” teaching seeks to end the divide between traditional and online instruction: By blending approaches, colleges hope to save money and meet students’ needs. Retrieved from http://chronicle.com/article/Hybrid-Teaching-Seeks-to/18487.