Predefined datasets
I'd like access to predefined datasets that provide ideas for my research and make it easy to get started.
This falls as a more-involved way for Twitter as a cooperation to support academic research on its platform (rather than just extending API limits for .edu emails etc..) but one avenue for very supportive, engaged involvement would be curation and publication of topics.
That is to say, imagine Twitter itself publishes corpora of tweet IDs corresponding to major events like natural disasters, political events, etc... This creates a streamlined way for those studying the social dynamics surrounding major events to implement real-time study of ongoing events, and broaden the scope of existing analyses.
Currently, the study of past events relies on hydrating tweet IDs from previous major events all compiled by data-horders and college librarians and archivists. And because of the non-trivial steps involved in accessing the data, to my knowledge there are only about 15-20 examples of 'high quality' datasets which would be suitable for use in any major, longitudinal study. Twitter leading the way in publishing these events would serve not only as a long-term marker in history, but as a reference for researchers the world over.
There are certainly individuals out there who would rather compile their own topics-- this is a rich research area in itself-- however there are many disciplines where the current barrier to entry for studying the twitter discourse around an event is prohibitively high. A Twitter-curated list of Tweet IDs surrounding an event (perhaps compiled based on keywords, location, and networks properties?) could be shared as a de-facto resource in the Online Social Network field.
It would represent a significant effort on Twitter's part to actively maintain and curate these databases. However it would go beyond what any other social media site has ever done in terms of supporting academic research.
AdminEmily Sheehan
(Product Manager, Twitter)
shared this idea
4 comments
-
Ricardo Correia
commented
I agree this would be a fantastic resource. This would also provide a standardised baseline dataset that could be used for example for machine learning or natural language processing applications. These datasets have been compiled in the past by researchers for specific purposes, but if Twitter were able to provide datasets covering specific topics or languages that are freely available, this would be a great resource for the community going forward.
-
Benjamin Rachunok
commented
This falls as a more-involved way for Twitter as a cooperation to support academic research on its platform (rather than just extending API limits for .edu emails etc..) but one avenue for very supportive, engaged involvement would be curation and publication of topics.
That is to say, imagine Twitter itself publishes corpora of tweet IDs corresponding to major events like natural disasters, political events, etc... This creates a streamlined way for those studying the social dynamics surrounding major events to implement real-time study of ongoing events, and broaden the scope of existing analyses.
Currently, the study of past events relies on hydrating tweet IDs from previous major events all compiled by data-horders and college librarians and archivists. And because of the non-trivial steps involved in accessing the data, to my knowledge there are only about 15-20 examples of 'high quality' datasets which would be suitable for use in any major, longitudinal study. Twitter leading the way in publishing these events would serve not only as a long-term marker in history, but as a reference for researchers the world over.
There are certainly individuals out there who would rather compile their own topics-- this is a rich research area in itself-- however there are many disciplines where the current barrier to entry for studying the twitter discourse around an event is prohibitively high. A Twitter-curated list of Tweet IDs surrounding an event (perhaps compiled based on keywords, location, and networks properties?) could be shared as a de-facto resource in the Online Social Network field.
It would represent a significant effort on Twitter's part to actively maintain and curate these databases. However it would go beyond what any other social media site has ever done in terms of supporting academic research.
-
Igor Brigadir
commented
One example that comes to mind where this was done really well in the past is https://blog.twitter.com/engineering/en_us/a/2015/evaluating-language-identification-performance.html more of the same kind of thing on various topics beyond language identification would be great
-
Geoff Bacon
commented
This is a really great idea, esheehan. One of the reasons I like this idea is that it would allow researchers to build out the rest of their project before having to pull data from the API. For example, I could use such predefined datasets to refine my preprocessing steps on realistic Twitter data before deciding what data I'd like to pull. Being able to play around with real data is always a great way to come up with new ideas on questions to ask.