Finding the Most Published Author in the New Yorker

The New Yorker is one of the most prominent publications of short fiction in the world. Among the authors published within the magazine have been some of the most famous of the last half a century including Haruki Murakami, Stephen King, Shirley Jackson, J.D. Salinger, Philip Roth, Alice Munro, and Valdimir Nabokov.

Spurred on by this thread in /r/AskLiteraryStudies, I thought it would be a neat little exercise to quantify who the most published author in the New Yorker was. There’s no public tally so I thought we should remedy that.

Data

Since the 90’s the New Yorker has been publishing on their website, but their archive of short fiction goes back all the way to 1925 which is amazing. All of their stories can be found at /magazine/fiction.

I built a little Beautiful Soup scraper to go through all the pages of the archive and retrieve a few pieces of data: author, title, link, description, issue, date published.

The full data set can be found here.

Some brief notes about the data:

• The data set contains all the stories published between March 4th, 2019 to February 21, 1925
• Most of the stories do not have descriptions as they only started to consisting put those in their stories after 2014

I hope the data set can help anyone interested in further studying the history of the New Yorker.

What I Found:

In total there have been 13,290 stories published amongst 2,327 authors. Over 50% of these authors only published one story within the pages of the New Yorker.

For the most published authors here is a list of the top ten:

Author	# of Stories
S. J. Perelman	271
James Thurber	270
John O'Hara	226
Frank Sullivan	191
W. E. Farbstein	184
E. B. White	179
John Updike	169
Sylvia Townsend Warner	152
Arthur Kober	151
Wolcott Gibbs	142

Futhermore here’s a visualization of all the authors who have published ten or more stories in the New Yorker:

What we can see is that S.J. Perelman and James Thurber are far away the most prolifically shown authors in the New Yorker’s almost 100 year history. But there are many, many interesting names in this top list. It would take a life time to try and appreciate all the different authors that have appeared on their pages.

Further Thoughts

There’s lots that can be done with this data set. My initial thought was to do some type of clustering with the story descriptions to see if there were any interesting insights there. But with the data set so small (most of the stories don’t have descriptions) I didn’t want to waste too much time there. Another would be try to download the entire corpus of stories and do analysis there, but I felt that might be against the spirit of what the New Yorker would want people doing with their site.

Other simple things that could be done: track publishing rates of top authors over time, automate finding author genders or nationalities to fill out the data set, show the change in the number of stories published over the years etc.

Anyways, hopefully this has been interesting. I think the data set would be useful for anyone that wants to use this as a jumping point for more research into the New Yorker, enjoys literary history, or just wants a more tabular form to browse through the thousands of stories the magazine has published.

Thanks for reading!