From Eureka

I made a couple of minor aesthetic edits unilaterally: I hard-coded the spacing of the first column in the first nested table so that "First Published wouldn't line break. That looked awful. Also, I chanegd the editors section so that the heading, and lists of editors on on separate lines. That looked better to me than having a whole list of editors cramped into half the column. --Ben Webster 15:54, 17 August 2007 (EDT)

Thanks! It's great fun to click on the first version of the page in the history tab, and then to repeatedly click "Newer revision" and watch how the template improves in real time! By the way, I think "unilateral" edits are the way to go; we can always revert if someone makes a boo-boo. --Brucebartlett 15:59, 17 August 2007 (EDT)

It would also be welcomed if someone were to make a completely new table from scratch. Such a template could be called Journal Info Prototype B, for instance. This way we avoid getting into a local maximum restricted by some fundamental design flaw of the initial table. --Brucebartlett 16:06, 17 August 2007 (EDT)

Contents 1 Data field on self-archiving missing 2 AMS data can be displayed in a collapsible box 3 Impact and citation statistics 3.1 Good news : www.eigenfactor.org gives a 3.2 My current thoughts : "Don't display citation rankings" 3.3 New thoughts on citation statistics? 3.4 concept : Chart for AMS data? 3.5 Volunteers needed : Turning the AMS data into charts 3.6 Banff protocol and Eureka

Data field on self-archiving missing

I forgot to add a field on whether the journal allows you to self-archive or not. This is important information :-) John mentioned the ROMEO data at the n-category cafe. Tom has also brought up related issues here. I think we should ultimately develop our own system... which can be modified and updated as time goes on, that's the beauty of a wiki!

I'm a little bit puzzled by the ROMEO data. Most of the journals seem to be given the "green light", for instance Topology. I believe the data is also a bit out of date, though I'm not sure. Someone should devise a tighter definition of the categories. --Brucebartlett 12:34, 18 August 2007 (EDT)

Yes, I agree that info on the journal's self-archiving policy would be great. (And by the way, I like the look of the prototype a lot.) Incidentally, it was a while before I understood the phrase "self-archiving" - it sounds like you're archiving your self! Seriously, I only just got its meaning, so I think we should either switch to a more self-explanatory phrase (though I can't come up with anything better right now) or be careful to provide an explanation.

It would also be good to record something about the type of legal agreement: do you have to transfer copyright? Most journals require this; I believe some (including AMS journals?) give you the choice; some only ask for a licence to publish. This could be covered as follows: "Copyright transfer? Compulsory/voluntary/no". --Tleinster 22:18, 19 August 2007 (EDT)

AMS data can be displayed in a collapsible box

I propose we use the nifty collapsible box widget to display the AMS pricing data. Just need to get javascript enabled at this site; shouldn't be a problem. Maybe display the last two/three years, with the rest initially hidden, unless you click "Show". The AMS data is paradoxical in that, while they display the no. of pages for each journal in 2005, for 2004 they say

2004 pages will be posted in latter part of 2006.

Does anyone understand this?

Could also have a tab which, when clicked, shows a tiny graph (in the template) with Price per page on the y-axis and years on the x-axis. Lol! --Brucebartlett 17:02, 19 August 2007 (EDT)

Impact and citation statistics

Do we really want to include the impact and citation stats? This quickly leads to the question: what is this site for? If it exists principally to push forward reform in the journal system, I don't see how these stats are relevant. In any case, many people dispute the significance of impact stats, and citation analysis seems to be an ever-more-common practice of highly dubious quality.

My worry is this. Suppose this site becomes very successful; suppose it becomes the canonical source of information on mathematics journals, the one that everyone goes to (like Google). Then everyone will see these "impact" stats, and they will become important. Perhaps they will assume an importance that they don't deserve. I suppose what I'm saying is: we shouldn't give high prominence to data of questionable significance. By all means let those clever little bots harvest the data, but I think we should ask ourselves whether we want to include it so prominently, or at all. --Tleinster 23:06, 19 August 2007 (EDT)

Fair point. You've hit on the problem of 'mission creep'. We should discuss this. We could display it as a link, as a plain-joe attribute, like this:

Impact analysis :

www.eigenfactor.org

Clicking on that link looks up the journal (in this case, K-Theory) at www.eigenfactor.org. Could possibly get it to display the pop-up (what happens if you click on K-Theory in the site) as well.

What about the www.journalprices.com data? They're economists, so their whole point is that free journals often have "more bang to the buck" (as they put it) than expensive journals. Here "bang" means citation indexes; otherwise it doesn't make sense to talk about "Price per citation". We could choose to exclusively show pricing data, with links to citation-related data... but I'm going to leave this debate to the more experienced mathematicians at this site (i.e. not me), since it revolves around the value of citation data, which I have no experience of. --Brucebartlett 09:35, 20 August 2007 (EDT)

I'm not precluding the possibility that we leave out anything that smells of citation data completely. In other words there are three broad options :

• Display everything we can get our hands on.
• Display pricing data, with links to citation data.
• Just display pricing data.

There is a very reasonable argument to be made for "just display pricing data". It keeps the focus on this site undiluted - to push reform in the journal system. This issue lies at the heart of the concept of this site : are we a "one-stop resource for the mathematician needing information on journals", or a "site pushing reform in the journal system", or a combination of both? --Brucebartlett 10:18, 20 August 2007 (EDT)

I've got to say, I think this is important information, and it should definitely by kept. If you want to start talking about mission creep, you should read my original post suggesting this wiki. You'll notice that there's not much in there about journal prices. Now, I'm not saying this by way of complaint, just to point out that people on this website are interested in make a variety of different sorts of information about journals available. That's the point of a wiki: Bruce and John can work on journal prices and I can work on other aspects of journals in a cooperative way on a wiki more easily than if we just tried to start a website.

Now, I agree that there is a danger in the false precision of numbers like Eigenfactor's, but I feel like it is the lesser of a number of evils. PageRank is a pretty good algorithm (certainly MUCH better than "just count citations" algorithm of the citation index). I also think it would be bad to not try to give some indication of which journals are high quality, and which are not. In the unlikely circumstance that this site becomes incredibly popular, and its use of eigenfactors numbers has a significant influence on people's assessment of journal quality, is that really so bad? Journals that publish articles people read and cite benefit, and since journals are really just important as signals anyways, as long as everyone is reading from the same playbook, everyone will be just as well off as before.

I suggested a site like this one, not because I'm pissed off about journal prices (though I am. It's just that journalprices.com already does a pretty good job of that), but because I'm pissed off about how confusing the journal system is, and thought young mathematicians could really use a source of information on journals, so they could have some hope of figuring out where to submit their papers. Prices is certainly one part of that; getting people to just note the price of journals they submit to would probably make a big difference. But journals have a very frustrating lack of transparency on a lot of other fronts, and I hoped a wiki would be a good place to try to bring that. --Ben Webster 14:06, 20 August 2007 (EDT)

Here's a slightly edited copy of an email I just sent Bruce.

Sorry not to have contributed further to this debate. After posing the question I thought I'd stay quiet for a bit to let other people say what they thought, and then for the last few days I've been away (and I won't be back for another week). However, now that you've nudged me, I'll try to reply.

1. I still have queasiness about numerical measures of quality (in almost any context). This holds especially in situations where people are liable to grab hold of them and interpret them as if they were the be-all and end-all. For instance, the UK has school league tables, in which schools are totally ordered according to certain quantities - exam results, etc. There may be good things about this, but one bad thing is that it encourages you to focus on the chosen factors and ignore other factors.

One might hope that mathematicians etc would be reflective enough and well enough educated in the pitfalls of statistics that this wouldn't happen.

2. I accept Ben's point that some measures of influence are better than others. An analogy was drawn between different ways of measuring the impact of papers and different search engine algorithms. Of course, the "better" the statistic is (whatever that means), the happier I'd be to see it included. The "why eigenfactor" page is impressive, for instance.

3. Re the little exchange on "mission creep", I don't think it matters one bit what the original idea was. We get to do whatever we want to do!

4. Given that it's a wiki, people can add whatever information they want. I may not volunteer effort on impact factors etc, simply because I'm less interested in that aspect of the site. But if other people think it's a valuable thing to include, then the way it works is that they can! And I think I've already stated my misgivings as clearly as I'm able to, so it's really up to them. That's not said bitterly; I mean it.

So I suppose I'm sitting on the fence. And the bottom line is point 4.

--Tleinster 20:57, 27 August 2007 (EDT)

Good news : www.eigenfactor.org gives a

I emailed Carl Bergstrom today, one of the people behind [www.eigenfactor.org]. You'll (i.e. the chief contributors here) hopefully be forwarded his full reply soon. In short, he was kind and encouraging. He would be happy for us to use eigenfactor data on the site. He will double-check with his colleagues about how this should work. About the idea of the eigenfactor statistic, he had this to say:

> Our initial motives in creating Eigenfactor were closely related: to help scholars, librarians, etc., better assess the value provided by scientific journals. One of the most common criticisms of my work on journal economics has been that I was relying on Impact Factor as a measure of quality, when clearly it has many flaws when used for economic analysis. This is a fair criticism, and one that I worried about a great deal. We developed Eigenfactor in an attempt to remedy as many of those flaws as possible, and give us a better metric for our work on journal economics.

Here is what he had to say about the issue of whether Eureka should list citation impact factors:

> Finally, with respect to whether Eureka should list influence rankings at all:

> My personal feeling is that if one wants to quantitatively evaluate the price of a journal, one needs to know what one is getting for that sum. Counting pages or articles gets one part of the way there, but not all of the way. One also needs to know whether the journal is any good and whether it is important in the network of scientific literature. My sense is that you need some influence metric to answer this latter question.

I'm adding this in as a contribution to the debate above, about whether we should display impact statistics or not. -- Bruce Bartlett

My current thoughts : "Don't display citation rankings"

I wrote this section earlier, before I read Ben's comment above. Well, he makes a good point, and its certainly in-line with his original post on the idea of a MathJournalWiki. The truth is, I'm confused right now and I just don't know what to think! I'm going to leave it for a day or two.

Here's the reasons why I sort-of convinced myself today that we shouldn't directly display impact factors (though possibly as a link):

• I can sleep easier morally. I'm terrified by the scenario Tom outlined, wherein the site becomes very popular and the canonical source of information about mathematics journals. A large part of that popularity might be due to the easy ability to browse and compare impact factors of journals. But deep down, I agree with Tom : I don't like the concept of "impact factor". I'd hate to go to my deathbed knowing I was one of the people who founded the site which promulgated and reinforced the idea of an impact factor. That scares me.

The paradox is... on the other hand, that's kind of the idea of a math journal wiki. So this is confusing.

• I can sleep easier legally. Not including eigenfactors, journalprices.com citation statistics, etc. will mean there's nothing that the publishers can nail us with. We have the freedom to do what we want. After all, it's a vicious circle : Springer pays Thomson to manage its citation data (or the other way round, I don't know!), we use data derived from Thomson's data... and use it to criticize Springer, so someday someone might get unhappy and start filing lawsuits. It's ridiculous : we have to pay people to find out who are the people we cited!!

• I can sleep easier literally. It makes the database simpler, easier to manage, and more focused. All those floating point numbers make my head twirl. It will probably mean that Eureka Science Journal Watch won't be as popular as it could have been, but that might be a good thing.

Anyhow I'm going to stop thinking about this for a day or two and leave it to other, wiser (i.e. people who've at least graduated), people to discuss this issue. --Brucebartlett 20:54, 20 August 2007 (EDT)

I think we should operate on the principle that this site will become very popular. Actually, I really think that it will — though it's possible that I'm just on an endorphin high from editing web pages. Operating on this principle doesn't mean that every decision we take has to be tremendously weighty, because — as Ben points out — it's a wiki! On the other hand, the decisions we take at this early stage are likely to have a big influence on the direction that the site goes in. --Tleinster 21:23, 20 August 2007 (EDT)

New thoughts on citation statistics?

Since there haven't been any further contributions to this thread, we are clearly all reflecting deeply over the nature, purpose, future and legacy of this wiki. In the spirit of this reflection period, I would therefore ask a clarification question from Tom.

Tom, what is it that bugs you about impact statistics? Is it that you regard the JCR Impact Factor (the total number of citations a journal receives, divided by the number of articles) as a dubious statistic in evaluating mathematicians and their papers? Or is it that you have issues with the actual notion of evaluating a mathematician's worth by placing some concocted number on his head?

And why does my feeble english instinct urge me to remove the question mark at the end of that last sentence? --Brucebartlett 08:39, 23 August 2007 (EDT) --Brucebartlett 15:26, 9 November 2007 (EST)

Of course, I'm not Tom, but for the sake of devil's advocacy, let me summarize my concerns about using Eigenfactor numbers. To me, the biggest is the false sense of precision given by a ranking, or numbers. People see that AMERICAN JOURNAL OF MATHEMATICS is 9th and ADVANCES IN MATHEMATICS is 11th and somehow think that this really means that the former is a better journal, when the difference is well within the "margin of error" (whatever that means in this context), when citation popularity isn't a metric of journal quality, it's a proxy for it.

The second is that we don't even know that Eigenfactor's algorithm is the best way of measuring citation popularity (and no real objective way of deciding what best would mean here). Now, I think PageRank is an excellent algorithm, and I trust Eigenfactor's numbers a lot more than the Impact Factor, but that doesn't mean it can't be improved on, and perhaps we don't want to contribute to Eigenfactor getting too established before people have had a chance to create competitors.

On the other hand, I think these are outweighed by two considerations: site users should have some way of comparing the quality of journals (and Eigenfactor is the best choice I know of; it's not perfect, but it really does measure one sort of value in a journal, and citations seem like the best way of collecting "the wisdom of the mathematical crowd") and the pernicious influence of the Impact Factor really does need to be combated (see, for example, this editorial), especially because impact factor systematically discriminates against mathematicians, which is much less of a problem with Eigenfactor.

As for what action should actually be taken...I wouldn't might moving the citation info lower in the template (I was the one who moved it up, and was mostly just messing with things to see how they looked), and I assume if Tom found leaving the info on the template really objectionable, he would have argued some more against it. Maybe we should elicit the opinions of the other people on the site? Blake, John, you don't seem to weighed in on this. Any thoughts? --Ben Webster 10:04, 24 August 2007 (EDT)

I like the idea of including Eigenfactor numbers. If Eureka becomes influential enough to popularize a ranking statistic, then surely it will be influential enough to spread concerns about those statistics. Blake Stacey 13:43, 25 August 2007 (EDT)

There's a blog post by John Ewing, Executive Director of the AMS, on impact factor. It's essentially a call for people to send their comments to the AMS — it's one worth keeping an eye on. --Tleinster 15:14, 4 September 2007 (EDT)

Another website for ranking journal is: www.journal-ranking.com They also use the page ranking algorithm. Interestingly, mathematics performs very well. If you include information on several ranking systems, you will not be seen as identifying one method as the leading method. - Stephen, 7 November 2007

Ok, thanks. It is an interesting site but I have to admit for some reason it gives me the creeps; perhaps because it's run by a commercial company, RedJasper. It isn't very easy to find out exactly what their "Journal Influence Index" means, even after I looked at the pdf file. --Brucebartlett 15:26, 9 November 2007 (EST)

concept : Chart for AMS data?

I added a possible chart functionality for the AMS data, together with some show/hide functionality so people can see the kind of things we can do. How should we display this data?

Remember in many cases we have journal price data going back to 1994/5, somehow skipping out 2005 for an odd reason. --Brucebartlett 20:01, 28 August 2007 (EDT)

Volunteers needed : Turning the AMS data into charts

There's some drudge work that needs to be done; any volunteers? Go to the page Volunteers needed : turning the AMS data into charts. It can be done in your own time; the charts will automatically get added to the journal's pages (due to coding in the template) when they get uploaded to Eureka.

--Brucebartlett 19:17, 31 August 2007 (EDT)

Banff protocol and Eureka

Lol, while thinking about the debate above about whether we should include impact factors, I thought about the following. Consider the Banff protocol:

We agree neither to submit to, referee for, nor participate in the operation of any journal that   
charges an excessively high per page subscription fee, as compared to the average of the 25
highest impact journals in pure mathematics.

One thing the semantic data in Eureka could do is display a live page showing the current "Banff boo-boo journals", those journals which people who've signed the Banff protocol shouldn't be doing stuff for :-) --Brucebartlett 14:04, 20 August 2007 (EDT)

Sounds great. That info should probably also appear on the journal's individual page (and if you want to get really advanced, it should appear on an editor's page whether they edit for Banff-incompatible journals).

Though, here's a question: the top 25 journals by which measure of impact? --Ben Webster 14:09, 20 August 2007 (EDT)

By the JCR Impact Factor, I think. On the other hand, "Article Influence" is sort of a similar statistic, and it's a more in-depth calculation. See the stats pages at eigenfactor.org to see the graphs comparing them. In fact, I just realized I was misinterpreting that graph to mean that they were sort-of proportional... but that's not true, is it? That graph alone doesn't give enough information, no?

For instance, the top 5 maths journals according to Article Influence are:

• Journal of the AMS
• Acta Math.
• Inventiones Mathematicae
• Ann. Math.
• Memoirs of the AMS

In 2004, the top 5 math journals according to Impact Factor were:

• Journal of the AMS
• Acta Math.
• Inventiones Mathematicae
• Ann. Math.
• Commun. pure and applied Math.

So... Memoirs of the AMS doesn't even feature in the Top 25 IF's. However, Commun. pure and applied Math is, at least, 6th on the list of Article Influence. In other words : different numbers. We'll have to put that in our pipe and smoke it.

I think it would be way too harsh though if a tag were to appear on editor's pages, "Warning : This editor has broken his declaration of the Banff Protocol." Lol! Let him that is without sin cast the first stone... --Brucebartlett 19:38, 20 August 2007 (EDT)