Community Discussions
Explore the latest discussions and community conversations related to this domain.
numpy - What is the "__.data" in python? - Stack Overflow
Main Post: numpy - What is the "__.data" in python? - Stack Overflow
Open __data files in Unity
Main Post:
Is there a way to open files from other games in Unity? I have a game with the assets stored in files named "__data", and I'd like to open them in Unity to do some editing. I'm really new at this platform, so I'm not sure if it's even possible.
Top Comment: The “files” are usually either models, textures, scripts and or other respective file types specific to the game (or engine). In almost every case they are referred to as “assets” not “files”. Some assets can be extracted and then imported by other game engines. There are multiple ways to do it (just search). As a side note, the “_data” folder that you’re referring to probably holds data within files (simple data containers, like JSON) as opposed to any actual assets from the game. Hopefully this helps answer your question a bit.
Question about Weblio
Main Post: Question about Weblio
r/Entrepreneurs on Reddit: Would you use a tool that validates your SaaS idea using Reddit data?
Main Post: r/Entrepreneurs on Reddit: Would you use a tool that validates your SaaS idea using Reddit data?
Presenting open source tool that collects reddit data in a snap! (for academic researchers)
Main Post:
Hi all!
For the past few months, I had discussions with academic researchers after uploading this post. I noticed that sharing historical database often goes against universities' IRB (and definitely the new Reddit's t&c), so that project had to be shutdown. But based on the discussions, I worked on a new tool that adheres strictly to Reddit's terms and conditions, and also maintaining alignment with the majority of Institutional Review Board (IRB) standards.
The tool is called RedditHarbor and it is designed specifically for researchers with limited coding backgrounds. While PRAW offers flexibility for advanced users, most researchers simply want to gather Reddit data without headaches. RedditHarbor handles all the underlying work needed to streamline this process. After the initial setup, RedditHarbor collects data through intuitive commands rather than dealing with complex clients.
Here's what RedditHarbor does:
- Connects directly to Reddit API and downloads submissions, comments, user profiles etc.
- Stores everything in a Supabase database that you control
- Handles pagination for large datasets with millions of rows
- Customizable and configurable collection from subreddits
- Exports the database to CSV/JSON formats for analysis
Why I think it could be helpful to other researchers:
- No coding needed for the data collection after initial setup. (I tried maximizing simplicity for researchers without coding expertise.)
- While it does not give you an access for entire historical data (like PushShift or Academic Torrents), it complies with most IRBs. By using approved Reddit API credentials tied to a user account, the data collection meets guidelines for most institutional research boards. This ensures legitimacy and transparency.
- Fully open source Python library built using best practices
- Deduplication checks before saving data
- Custom database tables adjusted for reddit metadata
- Actively maintained and adding new features (i.e collect submissions by keywords)
I thought this subreddit would be a great place to listen to other developers, and potentially collaborate to build this tool together. Please check it out and let me know your thoughts!
Top Comment: Thanks for this. I have to admit, I'm not familiar with IRB concerns around use of historical data. I've personally used PushShift for academic publication before, and have worked with students to build tools that mostly rely on PushShift for data capture. What are the concerns? How many people do the IRB limitations impact? There've been hundreds (thousands?) of papers published based solely on data from PushShift archives. I was sitting things out for the past ~6 months while the dust settled, so I'm happy to be corrected on this, but my personal sense is that 1) PushShift is still probably (by far) the most valuable way to collect Reddit data, via the archived/torrented files, 2) the Reddit API's restrictions are problematic for large data collection needs (i.e., the 1000 post limit), and 3) Hybrid is probably the way to go moving forward, where you pull from the archives and update as needed. But that's just my impression. That said, I've just this week started poking around the Reddit API to explore augmenting data (as in 3) above. So RedditHarbor is a welcome new tool to poke around with. Thanks for sharing!
How do I download reddit data?
Main Post:
I'm trying to download all Reddit data from the r/antiwork page at least from 2020 to early 2022, if not current. I'm an anthropology master's student and am new to this data analysis world and was hoping someone could point me in the right direction. How do people find/download this type of data? I'm trying to put it into R Studio, so excel or csv is best. Thanks, y'all!
Top Comment: There is a package called RedditExtractorR with some limited functions.
Best Methods for Scraping Reddit Data?
Main Post:
I'm working on a project where I need to send DMs to users from a specific Reddit community. Does anyone have tips on how to scrape Reddit data for usernames? Any tools or techniques you'd recommend would be appreciated
Top Comment: https://www.reddit.com/dev/api/ If you want to be responsible and not "antisocial"