Yahoo groups data update

[shadowscast]

Regarding getting the data from Yahoo groups:

I finally got the data which I had requested from Yahoo directly by following their not-very-clear "request your data" instructions (involving going through the "privacy dashboard"). (I requested the data on 2019-10-16 and I got it on 2019-10-26.)

It arrived as a zip file, which, when unzipped, had a folder for each of the groups I was a member of (not just the one I moderated).

Each of the group folders had three folders in it: files, links, and messages (also zipped, initially).

Unzipping the "files" folder yielded all of the files that had been in the group's files section, with subfolders intact. So that's pretty handy, I guess.

Unzipping the "links" folder yielded files which Windows tells me are of the type "internet shortcut", but which I cannot figure out how to open. Anyway, each one is about 200 bytes.

Unzipping the "messages" folder yielded one file with a name like: 2215412.mbox.00001
(Update: Whoops, when I wrote that I had only looked at the smaller group. For the larger groups, there are multiple mbox files, with the suffixes 00001, 00002, etc. Each one had a maximum file size of about 10,244 KB—so I guess the bigger message archives got broken into chunks, which makes sense.)

At first I wasn't sure what that was, but after some googling I figured out that it was a saved-emails file format.

I downloaded Thunderbird, an email client, and followed the instructions found on this page:
https://www.wintips.org/how-to-open-mbox-files-in-thunderbird/

After doing that, I was able to view all of the group's email messages in Thunderbird.

Update: After writing this post, I saw that there's now a Yahoo Groups Fandom Rescue Project Tumbler, which has a post with essentially the same information I just discovered for myself: Yahoo Groups Deletion: Requesting Your Groups

But they mention that not all files from the "files" section of the groups seem to be necessarily always included, so that's a warning.

And morgandawn has the same info in a Dreamwidth post: Yahoo Groups Deletion: Requesting Your Groups

Update 2: I think I've figured out what's up with the missing photos. See this comment below.

Comments

[morgandawn]

How many groups were you a member of? and how large was the single zipped file?

[morgandawn]

also some people found some Photos missing?

[shadowscast]

There were no files missing from the one group I checked in detail (but it was a small file section, with only 263 files in total).

[shadowscast]

Oh! Just realized...

When I grabbed the group's files using PG Offline, I got two different folders that contained photos. One was called "files" and contained various files members had uploaded, many of which were photos. Another was called "photos" and contained only image files.

This reflects the fact that on the group's homepage, there is a "files" tab and a "photos" tab, which are separate.

The Yahoo official data file gave me all of the stuff from the "files" tab but none of stuff from the "photos" tab.

In other words: the "files" folder I got from Yahoo was identical to the one I got from PG Offline, but PG Offline also gave me a "photos" folder, which Yahoo did not give me.

[catwoman69y2k]

Okay, this makes me feel better. In using the Yahoo tools (from the Get My Data section in Groups), I had the same problem where the zip file contained my files but left everything empty as far as the photos zip file (that was contained within). Question: What did PGOffline do regarding the photos' captions? Or is that something you are going to have to manually pair back (if you even want to) from one of the json files?

Since it seems more and more that PGOffline is the way people are going to have to go, would you have any background info on this tool as far as whether it could be trusted? One thing that has made me hesitant about what alteratives I implore is the fact that some of my groups' members are no longer with us and so I cannot get consent from all my members regarding the possibility of sharing or reposting the content. However, I *do* need to make sure I get all the photos before 12/14

[shadowscast]

What did PGOffline do regarding the photos' captions?

I'm not exactly sure what you mean by captions, but when I look at the photos at the Yahoo group page, each one has some kind of name given by the uploader (is that what you mean by caption?) plus the uploader's name and date. In the files saved by PG Offline, the photos' file names are the names given by the uploader, but the name of the uploader and the date are not attached to the photo in any way that I can see.

Note: if the photos had been organized into a folder within the Yahoo groups photo section (in the Yahoo interface they're called "Albums"), PG Offline will put them in a subfolder of that name.

Since it seems more and more that PGOffline is the way people are going to have to go, would you have any background info on this tool as far as whether it could be trusted?

What I can tell you is this:

1) I have downloaded and used it without any problems.

2) A lot of other people in fandom have reported downloading and using it without any problems.

3) The writer of the software is actively answering people's questions in his help forum, seems very friendly, and is aware that lots of people are now using the software to archive Yahoo groups before it's too late.

[catwoman69y2k]

One more question... were you ever working with PGOffline and a private group. This is one concern I have, as the groups Im trying to save did get put under the adult classification (as far as Yahoo policy at some of those evolutions in the timeline. ) I think that is how we became private. Realize that could be a problem for some tools (and Im wondering if thats somehow why photos was the one thing that Yahoo's data export couldnt retrieve for me).

[shadowscast]

My group that I used PG Offline for was restricted, meaning only members can see the message logs and files. PG Offline gets you to put in your Yahoo username and password so that it can acccess the group; the PG Offline software info tells you that the username and password are saved only locally, on your computer (and I have no reason to disbelieve it).

[shadowscast]

I was only in three groups. The single zipped file was 36,691 KB.

[yourlibrarian]

Although not everyone who opened or used a Yahoo Group was all that Internet savvy, I suspect that they'd still be better off than many people today trying to do all this.

[shadowscast]

Do you mean in the sense of understanding file types and directory structures?

[yourlibrarian]

Yes. I've had this conversation before, probably on my account as well, but there's been a marked difference shown in tech ability between fans in the YG period and the ones today.

It's not that everyone knew how to do things. I was just scanning through a group yesterday where for the first three months of its opening people were still trying to figure out how to transfer their subscription from another account, trying to figure out how to get email vs digests and other preferences, and so on. But I think that there was, perhaps, a greater likelihood of asking for help with problems as well as some basic stuff, and an effort to work things out.

For example, I'd be willing to bet that if you chose a random sample of fans who have been in a fandom for at least 5 years, at a minimum half of them wouldn't not know how to create an HTML link. I think that would have been much less true around 2000 or 2005 -- even though chances are the same number of people would actually understand HTML coding. You simply had to learn to work with the tech back then and be prepared to do a variety of things yourself. And because it was all computer based, you had to have some basic skills.

[shadowscast]

Although I lack any kind of data or even really anecdotes, I have a suspicion that you're right. Software nowadays doesn't want you to see its guts. (Well, Minecraft is a notable exception!)

I tend to forget that I'm really more tech savvy than most people—perhaps because I'm less tech savvy than a lot of the people in my close circles (but that's because my close circles are unusually techie!). But for instance I know how to build a web page from scratch, and I do bits of coding of one sort or another on a nearly daily basis.

[avrelia]

Wow! You did an amazing job. I am happy that I didn't have anything in Yahoo groups (I was a part of some but nothing worth saving).

But it does make me worry in general. Internet is such a weird place - things both stay forever and disappear without a trace here...

[shadowscast]

things both stay forever and disappear without a trace here...

That is the paradox of the internet.

Vid show Playlists

[(Anonymous)]

PGOffline stores all data (yours and the files) on your computer. But if you ever export the PGOffline database (this is the file with the pg4 extension), change your password. The files and messages and photos that you might share are fine ...but the database that PGOffline stores only on your computer contains your password. So change your password if you ever decide to hand over the database. In fact, I am changing my password at thee end of the Yahoo Group project -- because Yahoo's password security is for crap.

Re: Vid show Playlists

[shadowscast]

Thanks, that's good to know!