Twitter for Academic Research
- See what other ideas have been submitted and vote for ideas you like.
- If you have an idea that isn't already listed, create a new post.
- Include only one idea per post.
-
List of tweet response structure
Instead of returning two differents objects "data" and "includes" when calling endpoints like search tweets, it would be much simplier to have all the data inside one object.
Currently we have :{
data:[
{
// tweet 1
},
{
// tweet 2
}
],
includes:{
users:[
{
// user 1
},
{
// user 2
}
]
}better solution would be :
{
data:[
{
// tweet 1
// user 1
},
{
// tweet 2
// user 2
}
}4 votes -
Allow researchers to have more than one academic research project in progress at a time
Academic researchers often have multiple projects in development at one time, often surrounding the same topic and/or use case. Allowing researchers to have multiple research projects at once or to close out their single project and apply for a new one would help researchers to harness Twitter's data and derive insights more efficiently.
8 votes -
Allow academic research on deleted tweets/ suspended accounts
Twitter’s terms of service requires us to delete tweets we have collected when they are removed from Twitter. This makes academic research on certain topics practically impossible, as continually monitoring whether a tweet is still present is unfeasible, and much time could be spent on research that would to be have to have to be abandoned because using the data would violate Twitter’s terms of service. Could an exception requiring the deletion of tweets to the terms of service be made for academic research?
9 votes -
Provide a way to retrieve quote tweets
If I have a set of tweet IDs, I do not believe there is a natural way of collecting quote tweets of those tweets using the Twitter API v1 or v2. This limits the ability of me and other researchers to fully study things like political dialogue, polarization, and interactions with political elites.
I believe it may be possible to retrieve quote tweets by searching tweet text for mentions of the tweet IDs, since quote retweets technically contain a URL pointing back to the quoted tweet. However, I do not think that is obvious to many API users, even if…
10 votes -
Add the "sample" operator to v2 Academic Access
The sample operator that exists for Enterprise would be very useful for v2 Academic Access too: https://developer.twitter.com/en/docs/twitter-api/premium/rules-and-filtering/operators-by-product
2 votes -
Allow access to the "friends" of suspended accounts
I would like to study who networks of bots tend to follow, but since they are quickly suspended this data doesn't seem to be available anywhere.
8 votes -
Historically Accurate Friend, Follower, Status Count
It is often useful to know when an account gains new followers or followers new accounts. If a researcher queries a user, Twitter returns the number of followers or friends at the time of the query. That makes sense. If, however, I query a tweet, such as by asking for a user's 3200 most recent tweets, each of those tweets will show me the number of followers , friends, or tweets at the time of my query, not the time of the tweet authorship. The only way to get the true number of followers, friends, or statuses when a tweet…
6 votes -
Search Users powerful endpoint
I would love to have an endpoint which will allow me to search for accounts based on criteria like keywords in bio, location, min follower count, etc.
1 vote -
for research
Real-time Stock Predicate
1 vote -
Allow API access to tweets from suspended accounts
For research focused on still-existing troll farms, it's critical to have access to tweets of accounts Twitter already suspended.
As troll farms:
1) are highly skillful in pushing legit users to breaking Twitter rules (and therefore to become suspended)
2) frequently experience suspension of their own accounts
3) when creating new accounts, tend to refer to their previous accounts to easier regain followershipCurrently, conversations with heavy involvement of troll farm accounts look like this:
https://twitter.com/pj1m6y2z1fmwdn7/status/1151811700557045761This Tweet is from a suspended account. Learn more
This Tweet is from a suspended account. Learn more
(single live tweet)
This Tweet is…18 votes -
Add an API to retrieve Tweet threads
I request the Twitter to allow access to all the replies of a tweet (all the conversations of a thread). It would be very useful for academics working in user behavior analysis, argumentation mining, hate speech detection because comment or reply reflects the actual thought process of a replying user.
57 votesWe are working on defining ways of retrieving a thread using a specific endpoint.
In the meantime, you can use the full archive search endpoint to retrieve an entire conversation thread (https://developer.twitter.com/en/docs/twitter-api/conversation-id).
-
Informed Consent API
Implement an API / Bulk access call to notify users that their tweets were / are used in a study, with the option to opt-out, or opt-in to more information from the researcher. Maybe a DM with links to IRB / Home Institution / Researcher contact details?
Ideally, i'd like to notify and ask permission from people - in the past, i've implemented this as a Twitter list, adding people to give them some notification - but this doesn't give people an easy opt-out.
Given the number of times data is used, maybe an equivalent to the list of advertisers…
47 votes -
Make Twitter Labels visible in JSON payload
Twitter has launched some labels for Tweets that contain misleading information. For example, under those suspicious tweets, a label named as "Get the facts about.." will show up. However, these labels are not visible in the current json file. These labels are actually pretty interesting and valuable features because we can study the impact of it on the cascade of the misleading information, which will contribute to the current information cascading literature.
2 votes -
Predefined datasets
I'd like access to predefined datasets that provide ideas for my research and make it easy to get started.
This falls as a more-involved way for Twitter as a cooperation to support academic research on its platform (rather than just extending API limits for .edu emails etc..) but one avenue for very supportive, engaged involvement would be curation and publication of topics.
That is to say, imagine Twitter itself publishes corpora of tweet IDs corresponding to major events like natural disasters, political events, etc... This creates a streamlined way for those studying the social dynamics surrounding major events to implement…
25 votes -
New endpoint for detecting whether tweets are still live
If you have a large collection of tweets it can be unwieldy to use the API to detect which tweets have been deleted - beyond complying with Twitter's own terms of service this is also necessary to respect a users agency over their own content. When a tweet or tweets are no longer available can also have an impact on what and how the results of academic research can be reported.
When sharing a dataset of tweet ids, you can also spend a lot of time trying to hydrate tweets that have mostly been deleted.
It would significantly simplify this…
20 votes -
Nominate non-profit intermediaries for Firehose and other complete data APIs
Take part in Social Science One (https://socialscience.one/ ) and/or work with other non-profit intermediaries such as academic libraries and data providers (for example GESIS in Germany) for Firehose access or access to the complete follow network or other large scale data. They have the most experience when it comes to researchers' needs and can act as a trustworthy partner when it comes to protecting user privacy and the like.
21 votes -
Twitter Trends API
Many common research use-cases would be greatly facilitated by an API that could return summary level information. For instance, for any subset of tweets based on a particular query term(s) and period:
hourly/daily/weekly counts of matching tweets
histogram data of top hashtags / handles referenced / links shared
proportion of retweets / replies / first posts
proportion of dataset generated by top n contributors
These would be very valuable from a purely research perspective, but presumably they present considerable challenges for implementation. But a methodology similar to Google Trends might be possible.
12 votes -
We need an API to validate our bot/troll detection techniques.
What I would like is the ability to use an API to alert Twitter to the presence of malicious actors interfering in elections. Ideally, it would be process whereby we crowd source and gamify the detection pieces, with Twitter's decision to suspend (or not) as the ultimate validation determining who is best at the game.
Recently, I sent 500 Canadian election Twitter IDs our model saw as bots/automated to a third party scholar: Kurt Wirth <kurtawirth@gmail.com>. I asked him if I could share his response to what I sent, he said yes. His response is below:
---------- Forwarded…
7 votes -
Retrieve random tweet from the index (new API call)
Deploy a new API call to retrieve a random tweet from the index. Would be an important asset for characterization studies.
9 votes -
Bring back the Streaming APIs
Streaming was a VERY good+easy way to get low key sampling data, without digging into heavy duty "last x million tweets" data dumps. A couple of lines of code (ie minimum wasted researcher time) and you could start collecting/analysing data.
It is very much missed.
Not to mention, of course, the damage removing streaming has done to the general Twitter ecosystem: http://apps-of-a-feather.com/
7 votesStreaming endpoints have always been available, and are still available in the v2 Twitter API. The recently released Academic Research product track increases the access levels to this and other endpoints: https://twittercommunity.com/t/introducing-the-new-academic-research-product-track/148632
- Don't see your idea?