Electronic Dissertations Library

Exploring the development of the independent, electronic, scholarly journal, by Alison Wells

Methodology and method

The aim of the dissertation is to explore the extent of the electronic, independent scholarly journal. This will be achieved by scanning the NewJour archive for announcements of appropriate journals, accessing, and then categorising them. For the second part of the dissertation, which looks at readership of electronic journals, an e-mail survey will be used.

1. Creating a list of free electronic journals

The electronic journal is a relatively new tool in the possession of academics, some of whom have embraced it while others avoid it. This dissertation will be looking particularly at journals which are available free of charge to the reader. This is because it will be easy to look at each journal in detail, but also because there is a lot of debate on the funding of electronic journals in general, with free-to-read journals being espoused as the way ahead. Therefore, it is of interest to determine the current state of play in this area of publishing, and to evaluate the position and importance of the electronic journal in the scholarly communication process.

The NewJour mailing list started in 1992, and announces new electronic serials (not just scholarly ones, but newsletters, e-zines etc.) as they are alerted to them by the serial owner. It provides a brief summary of the aims and intended readership of the serial, and a link. The archive contains all the messages that have been sent to this list, currently >7000, and so is an excellent starting point for the study. An alternative method would have been to use a search engine to search for e.g "electronic journal", but this would have been too time consuming and frustrating as it would have brought up an enormous number of hits. The summaries given by NewJour are invaluable in weeding out likely journals from e-zines and other chaff, in addition, they also help to reduce the amount of time spent scanning the entries as they are all generally of the same format. To choose the journals that would be looked at in detail, I used the following criteria on each of the entries in NewJour:

1. Include immediately anything that uses the phrases "scholarly" or "peer-reviewed"
2. Discount immediately anything describing itself as an "e-zine", "magazine" or "newsletter"
3. Discount immediately anything published by the major publishers, such as OUP, Elsevier, Springer-Verlag etc. as they all make some charge for their journals, except where a journal is explicitly described as free of charge (and not just for a trial period).
4. Discount anything that describes itself as being "subscription only".
5. Discount electronic versions of print journals unless they are free and complete.
6. Make a reasoned judgement on anything not falling into categories 1-5.

Most of the journals entered in NewJour fell into one of the first five categories. I also added to this list by using other sources, such as the World Wide Web Virtual Library (http://www.edoc.com/ejournal/) and University of Glasgow library page (http://www.gla.ac.uk/Library/E-Journals/freejour.html), which had a useful list of free electronic journals. I was not sure how biased towards the US the NewJour site was, so I tried to use a couple of different lists to get a variety of journals. However, I found very few new ones this way, partly because these sites did not list dead journals, but also because the ones they did list had already been covered by NewJour. Because I wanted to measure the "death rate" of the journals, I made this preliminary list without examining the journal web sites, as I wanted to include all journals which had been alive at some point, whether or not they were now.

This was not a foolproof method as the statements had been written at the time the journal was announced, maybe up to five years ago, and quite often I found that some or all of the details had changed, for example the journal had changed address or no longer existed, or had become subscription only.

I decided to examine characteristics of the journals that could be determined reasonably quickly, merely by accessing them through the Internet, as this would mean that a large amount of data could be gathered in a reasonably short time. I could have e-mailed a survey with these questions, but then I would have been relying on a reasonable response rate, and for respondents to actually answer all the questions accurately, so I decided that it would be far easier to look at them myself. The characteristics were:

• Title

• URL

• Originating country : This was usually determined by looking at the country code in the URL, or in the case of an inconclusive URL, by looking at the country of origin of most of the editors.

• Peer reviewed? : I relied on the fact that if a journal was peer reviewed, then it would want to say so. Usually the journal would have a section with instructions for authors which would describe the review process if they had one.

• Organisation / person responsible for the journal : I selected this from the categories of Academic, Hospital, Learned Society or Professional Association, Other Group, Government, Individual or Commercial. It was usually easy to identify an endorsement along the lines of "Journal of the department of ... at the University of ..." etc.

• Broad subject : I tried to categorise the subject under the broad headings of Arts and Humanities, Social Sciences, Physical Sciences, Life Sciences or General. A few journals were interdisciplinary, but I put them under the heading of what I could see as the major discipline. For example I would put Journal of Technology Education under Social Sciences (Education) rather than Technology.

• Narrow subject : After assigning a broad subject area, I tried to define the exact subject heading. For both of these categories I used and adapted the subject lists used by ARL in their Directory of Electronic Journals.(http://ublib.buffalo.edu/libraries/e-resources/ejournals/)

• Format : I was keen to see how widely the differing formats for publishing on the Web were used. With few exceptions they all fell into the categories of plain HTML, Adobe Acrobat (.pdf), dvi and PostScript (.ps)

• Language : I attempted to list all the languages that the journal would publish in. However, the results of this will have been skewed by the fact that there had to be some English description so that I could firstly recognise it as a scholarly journal and then categorise it.

• Print primary or secondary : This recorded whether the electronic journal was merely a copy of a print version (print primary) or if it was an expanded version, or the electronic journal was printed in an annual archival version at the end of a year (print secondary).

• Year of first issue and year of latest issue : I used these two features to help establish the length of existence of the journal and also to decide whether it was "alive" or "dead", by using a cut-off date of 1.1.98, i.e. any journal which had an article published after that date was considered "alive", otherwise it was "dead".

• No of issues per year : I was interested to see whether the electronic journals were still keeping to the rigid paper discipline of volumes and issues, or if they were taking advantage of the medium and publishing articles continuously as soon as they were accepted.

• No of articles per year : A criticism levelled against many electronic journals in the literature was that they were not attracting a "critical mass" of papers, i.e. that people would not be interested in them if they only published four or five articles a year. I tried to record a rough estimate of the volume of papers published, usually based on the output in 1998. This was not a very scientific test because often the number of papers published varied greatly year on year, with some of the journals having had their heyday in the mid-90's, but tailing off recently. Also with journals first published in 1999, it is difficult to tell how many papers they will publish when they have become established, rather than just guessing from the first issue.

• Any other information : Here I recorded my own impressions of the site, whether I found the site design clear and useful, I also recorded other random pieces of interesting information.

From the original list, I found that many of the links were broken, so to try and find the journals I used the meta-search engine Dogpile (http://www.dogpile.com), and also any search facility on the host site.

2. Readership survey

As an area for more detailed study, the readership of these electronic journals is important to try and measure the impact that they are having on the academic community. I was also looking to see what the most reliable and also the most common way of assessing readership was. The number of journals advertising their hit rates on their Web site was very small, so it was not possible to get this information merely from looking at Web sites. I used a short e-mail survey to gain more information from the journal owners. I used an e-mail survey as I hoped that it would obtain a reasonable response, as people setting up electronic journals should be used to using e-mail, and also it is inexpensive to survey a large number of people to try and get a large enough response from which reliable conclusions may be drawn.

From the list of journals I had defined as still "alive", I selected all those which made some attempt to count their readers with one of these methods:

1. Setting up a mailing list which sent out a table of contents or abstracts from each new issue or a related discussion list
2. Asking readers to voluntarily sign in to the journal.
3. Making registration compulsory.

I tried to find an e-mail address for someone on these pages, either someone who looked after the Web pages, or a general enquiry address, or the editor, and sent them an e-mail asking how many people had registered for their journal, and also if they had any information on hit rates to the journal.

Although only about a quarter of the journals I had defined as alive fell into one of the three categories, I had quite a high response rate, although not all the respondents answered both questions, so feel that this was a successful method of getting readership information. I used the logfiles, either e-mailed to me, or available from the Web site to get information on hits. I also tried to get information about the countries of origins of the users and the referring pages to the journals, but as only a couple of journals had sufficiently detailed logfiles, I decided that no useful information could be gained on such a small sample.

The constraints on this study will be time (3 months), my foreign language ability (none), and the problems of running Netscape under Windows 3.11 on the University network.

The biggest problem I faced was that of time. It took a lot longer than I thought (about six weeks) to make the preliminary list and categorise the journals on it. This was due to the frustratingly slow speed of the University network and the Internet in general. I tried to do as much work in the morning as possible, as after about 2 p.m., it became nearly impossible to work on the Internet as Netscape (and quite often Windows) would need to be reloaded frequently. Most of the journals were easily categorised, however, a significant proportion failed to date any of their articles, or even give any indication as to when they had begun or were last updated. I think that this is a very great failure on their part, as the importance of knowing when something was written cannot be overstressed.

Title Page    Next section