Knowledge Management in Software Engineering : Discussion

By June 20, 2015

We now discuss our findings. We begin with a discussion concerning our two research questions, and end with a discussion of the validity of our study.
1 Concepts
In answering our first research question regarding concepts investigated empirically within the field, we decided to use Earl’s framework for schools of knowledge management. The final selection of papers was divided between the technocratic and behavioural schools, with an emphasis on the technocratic side. This was not surprising, given the general focus of software engineering on the construction of tools and processes. We did not find any examples of what Earl considers economic schools. The reason for this is probably that not many software companies track their intellectual capital.

Looking closer at the technocratic schools, we saw a heavy focus on the systems and engineering schools, with barely any mention of the cartographic school. The heavy focus on the systems school can be explained by the software engineering field’s focus on implementing new tools. The ratio of empirical versus lessons-learned papers also confirmed what has been pointed out previously; that there is a heavy focus on building new tools, but far too little on testing and reporting the actual usage of these tools. As mentioned previously, many of the excluded papers would have been placed in this category, had they had any empirical content. The main concepts we identified in this school were the development and use of knowledge repositories. There was, however, little to no overlap between the identified papers, which underlines once again the need for more empirical research.

The engineering school is the school that received the most empirical attention, according to our review. Again, we identified two main areas within this school: those focusing on the entire software process and those focusing on particular activities within the process. Within the papers focusing on specific activities, we identified four main areas: formal routines, mapping of knowledge flows, project reviews, and social interaction. As with the systems school, there is little or no overlap between the empirical studies. A possible explanation for the heavy empirical focus within this school is the close fit with work on the improvement of software development processes.

That there are so few papers in the cartographic school is interesting. One possible explanation is that the ”yellow pages” systems are considered ”simple” and undeserving of attention. However, as the lone study in this category shows, such tools have uses other than the obvious. This school could benefit from more studies of actual usage.

In the behavioural school, we found a limited number of papers focusing on organizational and strategic aspects, and no papers focusing on spatial aspects.
The three studies in the organizational school discuss the use of people networks in software organizations. Two of the studies investigated the improvement of software development processes. In Earl’s taxonomy, both intra- and interorganizational communities are mentioned as examples. In the software engineering literature, we only find studies made in single organizations.

As for the spatial school, no empirical studies on software engineering were found in this category. This is clearly an area where more research should be conducted. The role of open-plan offices has been studied in other fields, and this is something that also should have an impact on how knowledge is shared in software teams. Many of the agile development methods recommend open-plan offices.

The empirical studies in the strategic school focus on factors pertaining to successful knowledge management, learning processes, and types of strategy for managing knowledge. It was, perhaps, to be expected that there would not be many articles discussing the strategic importance of knowledge in software engineering supported by empirical findings, because its importance is assumed in most published works on knowledge management in software engineering.
2 Research methods
Of the 68 studies identified, 39 were reports of lessons learned and 29 were empirical studies. Case studies constituted the largest number of empirical studies (see Table 4), followed by field studies and action research. It is positive that the emphasis on empirical studies has increased (see Figure 1). The apparent dip in 2006 is due to the time at which the search was conducted. We searched the databases in August and most compilers of databases take some months to index their papers; hence, we can only claim to have covered the first third of 2006 fully.

The research methods in the studies that we selected are dominated by case studies, both single and multiple. This is not surprising, considering our limitation on only including studies that performed tests in industry.

Glass et al. [46] found that empirical studies constitute about 5% of published research in software engineering as a whole. Comparing our final findings to the results from our first rough sorting of papers, our final selection constituted about 3% of the initially selected papers. If we assume that Glass’s data are representative for the area that we studied within software engineering, we could extrapolate that about 70% of those papers would be conceptual analysis and concept implementation. Most of the papers discarded were indeed conceptual analysis and concept implementation without empirical testing, our results do however, not show a discard number on the empirical criterion as high as 70%. Many studies were also excluded because they were not relevant to either software engineering or knowledge management. Therefore it seems that empirical studies constitute a larger part of the studies on knowledge management in software engineering than in software engineering in general.
3 The state of research on knowledge management in software engineering
We identified far more studies, particularly empirical studies, than have been reported in previous assessments by Rus et al. [89], Lindvall [72] and Dingsøyr and Conradi [32]. We have also shown that although there are not many empirical studies, except for in the systems and engineering schools, there are either empirical studies or reports of lessons learned in all schools except the economic school. Thus, research on knowledge management in software engineering seems to be slowly gaining a broader focus, although research on knowledge management in software engineering is still somewhat distanced from mainstream research on knowledge management.

If we compare the studies found in software engineering to the research directions suggested by Alavi et al. [3], we see that software engineering has primarily addressed the storage and retrieval of knowledge, while topics such as knowledge creation the transfer and application of knowledge still needs more attention.
4 Limitations
The main threats to validity in this systematic review are threefold: our selection of the studies to be included, the classification of studies according to Earl’s framework of schools in knowledge management, and potential author bias.

As for the selection of studies, only one researcher read through and discarded the first results on the basis of the papers’ titles. However, in cases where there was doubt, the papers were included in the next stage. The second and third stages, which were based on abstracts and full papers, were carried out by both researchers and we observed a ‘good’ degree of consensus. In cases where there was disagreement, the issue was discussed until concensus was reached.

Finally, there is a potential bias in that both authors have written papers that were included in the review. Where only one author had participated in the primary study, the other author decided whether or not to include it if there was disagreement.