Doing and writing a literature review is part of any academic process and it is important to master. You will do a literature review in many cases – both when writing academic texts (like a scientific paper, a PhD Thesis, or an MSc Thesis), when designing new software (like a new framework or an end-user application), and even when you’re working commercially and e.g. wants to patent something you’ve done.
Some of the following text is highly inspired and “stolen” from Saul Greenberg’s page on “How To Write A Literature Review“. I’ve opted to transfer the text here since Saul is retired and I don’t know how long his web server will keep running. In addition, I’ve added things on my own, especially on how to use search engines.
Why Make a Literature Review?
Everything in academia is about “making a contribution to science”.
The main purpose of a literature review is to be able to explain the contribution of your work as compared to others. If you don’t know the works of others, then you cannot explain your contribution – it is that simple!
The Nature of a Literature Review
Your job when writing a literature review is to add value to all those papers you have read, where you explain how the many salient ideas of others (often gathered from many disparate sources) have led up to and have contributed to your research problem. Let me say that again: a good literature review adds value. It is not just a catalog of papers you have read.
Perhaps the most critical point is that your literature review must frame your work. It will give the reader sufficient background to motivate your work, to understand what has influenced your work, what others have contributed to it, and – most importantly – how your work and the problems you propose to solve add to that literature.
What does a Literature Review do?
A literature review can play several roles in your thesis or paper:
- Show you know the literature
- but there is far too much literature for you to be exhaustive, so you must be selective.
- Gives your readers a background to understand your work
- this includes both readers who are specialists in your area, and readers who know nothing about it (e.g., external examiners). It’s a delicate balancing act.
- Gives a historical perspective
- shows how ideas arose and evolved over time.
- Leads to the problem you wish to tackle in your thesis
- what others have done before within this context,
- what is being done now,
- what problems have been identified,
- what has not been worked on, and how your own work adds to this.
- Describes related work
- illustrates other ideas related to your research idea i.e., how they are in common and how they are different.
- explains why your idea or perspective is new.
- Gives a new view of the problem or solution space. This can be done in three ways:
- Synthesis – combines together the literature in a way that adds something new i.e., the whole is greater than the sum of the parts.
- Bird’s eye view – as a reader of the literature, you now have a better perspective than any individual author may have had (particularly earlier authors who have not seen later work). You can now ‘step back’ and give a coherent overview of all that has happened.
- (Conceptual) Framework – fits all the pieces together into an organization that relates all the parts.
How to Find Relevant / Related Literature?
There are a number of different methods for searching and finding relevant literature. But before even starting to use these methods, it is important that you know what to search for! Hence, it is important that you know things like:
- What is it you’re researching, in order words, it is important that you have your own research questions and intended contributions in place (see the section on different Thesis Types on the “Doing a MSc Thesis” page).
- What concepts and names are used for what you’re doing?
- What scientific community are you addressing? For example, if are you working in computer science, the ACM is most likely the starting point; if you’re in software engineering then IEEE is most likely the starting point; and if you’re in a medical field, then different medical societies and journals are relevant.
Basically, it is important to know what you’re searching for (your own problem statement), where you will search for this (the scientific community), and what it is called (the concepts and names).
Once you have these things in place, there are a set of different methods for finding relevant scientific literature:
- Overview – first, try to get an overview of the concept you’re researching. Here, Wikipedia is a good source. For example, if you’re doing research in Mobile Sensing or using Participatory Design methods, Wikipedia has useful pages.
- ChatGPT – a recent addition to getting an overview is using ChatGPT. It normally does not give a comprehensive list of papers, but it provides an “essay” in which a set of (what it thinks) are key papers in the field. These papers can then be used for further search, using e.g. the snowballing method. See below for an example from ChatGPT.
- Search – the most common way to find papers is to search. Here Google Scholar is by far the most efficient search engine. But also the ACM Digital Library and IEEE Xplore are good search engines. If you’re in medical applications, PubMed is also a key entry point.
- If you’re a DTU student or employee, the DTU Findit search engine from the DTU library is very efficient too. In addition, DTU Findit will enable you to access papers that are not open access and requires a subscription.
- If you’re drafting up a patent, then Google Patents is a good starting point, but soon you will need to move to the professional patent databases which require a subscription.
- Survey Papers – in some research traditions (especially healthcare, but also increasingly in computer science) review or survey papers are being published. It is of course a tremendous help if there is a survey paper within your topic. Some scientific societies even have special journals for surveys, like the ACM Computing Survey journal.
- Snowballing – refers to a method where you find a couple of anchor papers and “snowball” your way through their references. You should identify a few key papers (e.g., 7–10), which you can do by asking your supervisor, sending emails to knowledgeable people in the area, or asking ChatGPT. Once you have these papers, you do both “forward snowballing” and “backward snowballing”.
- Backward snowballing – you look up the key references in an anchor paper. Typically, each anchor paper has a section called “Related Work” and this is a goldmine of relevant papers also for your literature search. This approach goes back in time from the publication date of the anchor paper.
- Forward snowballing – you look up which papers are referencing each anchor paper. This can be seen in Google Scholar, which for a paper shows “Cited by”, i.e., which other papers cite this paper. This approach goes forward in time.
- Systematic Literature Review – refers to the “state-of-art” method for doing a literature review. The “systematic” part refers to that you have a set of strict keywords you are using in your search term and that you have a set of well-defined including and exclusion criteria.
- The essential difference between a literature review and a systematic literature review is that a literature review provides evidence from a high-level summary of the fields connected to a research question whereas a systematic literature review begins with an intentional and purposeful selection of data including types of information to be included in the review including policy documents, journal articles, book chapters, blogs and publications related to the research questions.
- We did a systematic literature review on mobile sensing, which illustrates how this is done.
- There is even a YouTube video on how to do a systematic literature review (but also contains useful hints for all sorts of literature reviewing):
How to Organize a Literature Review?
Several literature styles are described below. Of these, the best literature reviews are usually organized around ideas and concepts. Use the other styles only if you have a strong reason.
An annotated bibliography is typically a list of papers ordered by some means (perhaps alphabetically or by topic), where each paper is represented as a reference plus a summary paragraph.
- a very useful way for you to track your own readings as you read them… it becomes an annotated index to your thoughts and ideas collected as you read papers
- for others, ideally, an annotated bibliography brings together papers from different fields or publications; this is especially valuable in new areas where the literature is scattered, or in areas that synthesize research from several other disciplines
- they are also very valuable for others seeking papers who wish to scan a relevant list of papers and summaries on a topic
- they are best when you sift out the chaff from the wheat i.e., if you include good papers and filter out the weaker ones.
- annotated bibliographies do not add value to the individual work i.e., they do not try to assimilate the material or generalize from it
- current electronic bibliographies with good search capabilities can automatically produce something akin to an annotated bibliography, especially if they include abstracts
- because they are just a ‘list’, they require constant updating as new relevant papers come out
Bottom line – a great way to annotate the papers you read while gaining a background in the area, but a poor way to do a literature review for a thesis
Summaries of Project and Paper
One common way to do a literature review (especially in systems-oriented research) is “by project”. For example, each sub-section may describe and summarize a particular project in detail and give the most significant references to it.
- lets you summarize and describe the most salient details about a project.
- you can collapse several papers into a single project summary
- you can describe the evolution of a project over time
- you can omit repetitive and early papers that are covered by later more mature ones
- it’s easy to get bogged down on project descriptions/details that have little to do with your main thesis interests; i.e., you may end up having to add much detail to set the context instead of concentrating on that part of the project/system that is of fundamental interest to you.
- some ideas are common across many projects and consequently, you are forced to either repeat yourself or introduce the idea in one project while only briefly mentioning it (or omitting it) in others
- if there are many projects/systems related to your work, you may end up writing a very long exhaustive chapter to get complete coverage; this may actually take away from the impact of your review
Finally, there will always be a system you missed, or another one just being built around the corner. Does this mean you have to keep adding descriptions? If you omit one, does this leave you open to the challenge in your defense about “how come you omitted System X” when (perhaps) you never heard of it or neglected to include it?
Bottom line – a reasonable way to describe the current state of the art if its fairly new, but can quickly become overwhelming.
Another obvious way to do a literature review is from a historical perspective.
- useful if you wish to show the evolution of ideas over time.
- only works if this evolution really does lead to your research problem. Thus you may have to consider carefully if this is the approach you should take as your main organizing principle.
Concepts and Ideas
A very good way – or I would say the best way – to do a literature review is by reviewing ideas, and then bringing in various projects and citations relevant to that idea. Ideas are typically abstractions that you may have made after reading many papers, project descriptions, etc. Ideas or concepts are the essences of what makes something work. Ideas may be presented either as a set of subsections of related ideas (where one will flow from the next), or even better as a conceptual framework that shows the inter-relations between concepts as well as their structure.
- you can concentrate on only the ideas salient to your research
- you can bring in other works and projects by showing how they relate to that idea i.e., so you do not have to go into depth on less relevant parts of those other projects. That is, you can emphasize key points of other systems rather than the unnecessary details.
- when you come across new projects or papers, you can examine how they relate to those ideas, and only include them if they add value
- if you are asked in a defense about a particular system or project that you had not mentioned, you can relate it back to your idea i.e., to say that you have in fact reviewed the idea covered by the system, and then show how that fits into it.
- requires thinking – unlike project summaries and annotated bibliographies, it’s up to you to distill the essence of what is important in your readings.
Bottom line – this is what you should strive for, as you are adding value by abstracting key concepts from the literature. But it does require some thinking! If you only bring out shallow ideas, your literature review will feel shallow as well.