Twitter for Academic Research
- See what other ideas have been submitted and vote for ideas you like.
- If you have an idea that isn't already listed, create a new post.
- Include only one idea per post.
-
Allow academic research on deleted tweets/ suspended accounts
Twitter’s terms of service requires us to delete tweets we have collected when they are removed from Twitter. This makes academic research on certain topics practically impossible, as continually monitoring whether a tweet is still present is unfeasible, and much time could be spent on research that would to be have to have to be abandoned because using the data would violate Twitter’s terms of service. Could an exception requiring the deletion of tweets to the terms of service be made for academic research?
7 votes -
Provide a way to retrieve quote tweets
If I have a set of tweet IDs, I do not believe there is a natural way of collecting quote tweets of those tweets using the Twitter API v1 or v2. This limits the ability of me and other researchers to fully study things like political dialogue, polarization, and interactions with political elites.
I believe it may be possible to retrieve quote tweets by searching tweet text for mentions of the tweet IDs, since quote retweets technically contain a URL pointing back to the quoted tweet. However, I do not think that is obvious to many API users, even if…
7 votes -
Allow researchers to have more than one academic research project in progress at a time
Academic researchers often have multiple projects in development at one time, often surrounding the same topic and/or use case. Allowing researchers to have multiple research projects at once or to close out their single project and apply for a new one would help researchers to harness Twitter's data and derive insights more efficiently.
4 votes -
Allow access to the "friends" of suspended accounts
I would like to study who networks of bots tend to follow, but since they are quickly suspended this data doesn't seem to be available anywhere.
6 votes -
Historically Accurate Friend, Follower, Status Count
It is often useful to know when an account gains new followers or followers new accounts. If a researcher queries a user, Twitter returns the number of followers or friends at the time of the query. That makes sense. If, however, I query a tweet, such as by asking for a user's 3200 most recent tweets, each of those tweets will show me the number of followers , friends, or tweets at the time of my query, not the time of the tweet authorship. The only way to get the true number of followers, friends, or statuses when a tweet…
5 votes -
Allow API access to tweets from suspended accounts
For research focused on still-existing troll farms, it's critical to have access to tweets of accounts Twitter already suspended.
As troll farms:
1) are highly skillful in pushing legit users to breaking Twitter rules (and therefore to become suspended)
2) frequently experience suspension of their own accounts
3) when creating new accounts, tend to refer to their previous accounts to easier regain followershipCurrently, conversations with heavy involvement of troll farm accounts look like this:
https://twitter.com/pj1m6y2z1fmwdn7/status/1151811700557045761This Tweet is from a suspended account. Learn more
This Tweet is from a suspended account. Learn more
(single live tweet)
This Tweet is…13 votes -
Add an API to retrieve Tweet threads
I request the Twitter to allow access to all the replies of a tweet (all the conversations of a thread). It would be very useful for academics working in user behavior analysis, argumentation mining, hate speech detection because comment or reply reflects the actual thought process of a replying user.
54 votesWe are working on defining ways of retrieving a thread using a specific endpoint.
In the meantime, you can use the full archive search endpoint to retrieve an entire conversation thread (https://developer.twitter.com/en/docs/twitter-api/conversation-id).
-
Informed Consent API
Implement an API / Bulk access call to notify users that their tweets were / are used in a study, with the option to opt-out, or opt-in to more information from the researcher. Maybe a DM with links to IRB / Home Institution / Researcher contact details?
Ideally, i'd like to notify and ask permission from people - in the past, i've implemented this as a Twitter list, adding people to give them some notification - but this doesn't give people an easy opt-out.
Given the number of times data is used, maybe an equivalent to the list of advertisers…
44 votes -
Make Twitter Labels visible in JSON payload
Twitter has launched some labels for Tweets that contain misleading information. For example, under those suspicious tweets, a label named as "Get the facts about.." will show up. However, these labels are not visible in the current json file. These labels are actually pretty interesting and valuable features because we can study the impact of it on the cascade of the misleading information, which will contribute to the current information cascading literature.
2 votes -
Predefined datasets
I'd like access to predefined datasets that provide ideas for my research and make it easy to get started.
This falls as a more-involved way for Twitter as a cooperation to support academic research on its platform (rather than just extending API limits for .edu emails etc..) but one avenue for very supportive, engaged involvement would be curation and publication of topics.
That is to say, imagine Twitter itself publishes corpora of tweet IDs corresponding to major events like natural disasters, political events, etc... This creates a streamlined way for those studying the social dynamics surrounding major events to implement…
24 votes -
New endpoint for detecting whether tweets are still live
If you have a large collection of tweets it can be unwieldy to use the API to detect which tweets have been deleted - beyond complying with Twitter's own terms of service this is also necessary to respect a users agency over their own content. When a tweet or tweets are no longer available can also have an impact on what and how the results of academic research can be reported.
When sharing a dataset of tweet ids, you can also spend a lot of time trying to hydrate tweets that have mostly been deleted.
It would significantly simplify this…
19 votes -
Nominate non-profit intermediaries for Firehose and other complete data APIs
Take part in Social Science One (https://socialscience.one/ ) and/or work with other non-profit intermediaries such as academic libraries and data providers (for example GESIS in Germany) for Firehose access or access to the complete follow network or other large scale data. They have the most experience when it comes to researchers' needs and can act as a trustworthy partner when it comes to protecting user privacy and the like.
19 votes -
We need an API to validate our bot/troll detection techniques.
What I would like is the ability to use an API to alert Twitter to the presence of malicious actors interfering in elections. Ideally, it would be process whereby we crowd source and gamify the detection pieces, with Twitter's decision to suspend (or not) as the ultimate validation determining who is best at the game.
Recently, I sent 500 Canadian election Twitter IDs our model saw as bots/automated to a third party scholar: Kurt Wirth <kurtawirth@gmail.com>. I asked him if I could share his response to what I sent, he said yes. His response is below:
---------- Forwarded…
7 votes -
Twitter Trends API
Many common research use-cases would be greatly facilitated by an API that could return summary level information. For instance, for any subset of tweets based on a particular query term(s) and period:
hourly/daily/weekly counts of matching tweets
histogram data of top hashtags / handles referenced / links shared
proportion of retweets / replies / first posts
proportion of dataset generated by top n contributors
These would be very valuable from a purely research perspective, but presumably they present considerable challenges for implementation. But a methodology similar to Google Trends might be possible.
10 votes -
Retrieve random tweet from the index (new API call)
Deploy a new API call to retrieve a random tweet from the index. Would be an important asset for characterization studies.
8 votes -
Bring back the Streaming APIs
Streaming was a VERY good+easy way to get low key sampling data, without digging into heavy duty "last x million tweets" data dumps. A couple of lines of code (ie minimum wasted researcher time) and you could start collecting/analysing data.
It is very much missed.
Not to mention, of course, the damage removing streaming has done to the general Twitter ecosystem: http://apps-of-a-feather.com/
7 votesStreaming endpoints have always been available, and are still available in the v2 Twitter API. The recently released Academic Research product track increases the access levels to this and other endpoints: https://twittercommunity.com/t/introducing-the-new-academic-research-product-track/148632
-
Advertisement API
Access data about ads, reach, accounts promoted by, engagements.
6 votes -
Consider Different Automated Account Regulation for Academics Running Studies with Bost
Bot-based research is an important frontier in several different academic fields, but post requests are often censored or accounts associated with such posts are often suspended-- even if they do not seem to violate Twitter's automated account policies. Perhaps there could be a mechanism where academics could petition twitter not to censor bots that are being used for academic research (assuming, very importantly, that they have been carefully vetted by IRB and/or ethics officers within Twitter)?
7 votes -
GDPR (General Data Protection Regulation) compliante stream
Doing research in europe is becoming harder, and GDPR add some difficult on exploiting data. I would like to know if data that we can get with the Stream API can be Anonymized.
5 votes -
Extract data without writing code
I'd like to directly access Twitter data without needing to write code to collect data from the API.
5 votes
- Don't see your idea?