Francine - harvest
I Blame the Dutch mpoetess
Previous Entry Share Next Entry
Oh LJ Oracle...
Is there a way to view only those entries of your own that you have set to a specific friends-filter, or have marked private? I'm trying to clean up my list of friends filters (which has a number of very similar filters that were created over the years, and don't really have a lot of difference between them now) and would like to be able to switch all entries posted on one filter to another filter -- I know there's no way to do that as a batch (at least not as a non-programmer) but it would be nice not to have to go through every entry ever, including the public ones, just to find the ones locked to a specific friends-group.

I thought (and so did maeyan, because we share a brain) of the workaround of putting one of my character/sockpuppet journals on whichever filter I'm trying to clean up, and only that filter, but that would still show all my public and friendslocked-no-filter entries, too.

Suggestions welcome; if there's an LJ client that is sortable this way, that would be fine too.


2005-08-05 03:25 pm (UTC) (Link)

There used to be a way. Years ago. Don't know if the programming still supports it.


You're going to expect me to remember how, aren't you?


2005-08-05 04:50 pm (UTC) (Link)

Nope - *pets your brain* - I figured out a (somewhat dirty) way, with Chicken's help.


2005-08-05 03:30 pm (UTC) (Link)

Well, I download all my entries in XML format for backup purposes, and I see that an easy way to find out which entries are security-locked in a certain way is to search for <security>usemask</security> and then winnow those down to the ones with a given <allowmask>.

But that would only let you know which ones were relevant (searching also for the associated subject/entry date) -- it would not allow for editing. Hmmm. Have you tried asking over on lj_nifty or lj_dev?


2005-08-05 03:40 pm (UTC) (Link)

Haven't asked on any of the official lj comms yet, as that always makes me nervous; figured I'd plumb the wisdom of my friendslist first -- thogh I probably will eventually, if I don't come up with something.

I'm not sure how one even goes about searching in XML, since I've never downloaded my LJ in that form. It would be fine to just find the relevent entries, as long as it gave me the direct URL for the actual entry online, but I don't know if the xml archive of the journal has that or not.


2005-08-05 03:57 pm (UTC) (Link)

Actually, it doesn't technically contain the full URL, but that's simple to reconstruct -- you'd just take the <itemid> and append '.html' to it, then append *that* to your standard URL. So if the <itemid> in question is 501178, you just tack on "" at the beginning and ".html" at the end.

I could write you a little script that could give you all the revelant links, actually.

Even better than reassembling the reading/browsing URL, I could reassemble it in editing mode, which is even easier since it would only require appending the <itemid> to the standard editing URL, e.g. taking "" and appending '501178' to the end. Huh. Piece of cake.


2005-08-05 04:05 pm (UTC) (Link)

Gotcha -- yeah, I don't need the full url - the itemid is actually enough.

Picking through the entries themselves one by one and changing them over will likely be ironically easy -- they're all filters I'm sure I didn't post more than a few entries to -- which makes them, of course, hard to find among the bunches of other entries, but easy to change over once found.


2005-08-05 04:24 pm (UTC) (Link)

I just checked, and the official LJ export tool is WAY more user-friendly than it used to be. Now you can download not only in XML format but also in comma-separated format, which is trivial to do a search & replace on to construct the URLs you want.

Just go here: and uncheck everything except "ID Number", "Security Mask", and "Allow Mask". Choose the CSV format, and put in date values, like "2005" and "07" for July. I did this for my journal and got back:


Then I'd do that for all the months I need, paste them all together in one big file. Then I'd search to weed out all the public ones with a 'grep -v'. After that I'd do two more greps to shunt all the private ones to one file and all the usemask ones to another file. Then I'd sort the the usemask file so that all the ones with given masks (filters) would be clumped together. Finally I'd do a little search & replace to transform these files into HTML files with the proper URLs as discussed in my previous comment. It would not take me long.

The beauty of this minimalist output is that it doesn't contain your actual entry, so if you wanted me to do it all for you, you wouldn't even have to expose your private stuff to me, because all you would send me would be a file containing the above-type data.

It's easy enough you could do it yourself, but I do this stuff all day for a living, so it would not take me long, I promise. It's up to you. Either way, the thing that will probably actually be the most annoying is having to download one month at a time. I wish their export tool let you just grab it all at once. And this is a task you'd have to do yourself since you have to be logged in to use the export tool.

While you are deciding and/or downloading, I can quickly prototype the workflow-y goodness using my own data as a guinea pig.


2005-08-05 04:30 pm (UTC) (Link)

Oookay.. you know you lost me the minute the word "grep" came out of your keyboard, right? *g*

I'm poking through the xml version now, and the one thing I'm missing is, how on earth do I know what mask is what, if they're all just numbers, and not represented by the names I actually gave the filters?


2005-08-05 04:37 pm (UTC) (Link)

Well, 'grep' is my search and sort tool of choice, but one can use any number of searching and sorting techniques to achieve the same end.

I do recommend the comma-separated instead of the XML in this particular case.

Also, as for the correspondence between the numbers and filter names, the LJ folks have been a bit crafty on that front.

We're talking "custom friends groups" right? ( If so, the "view source" on the HTML of the URL I just pasted does seem to embed a number in the value= attribute for the editing of the groups, but those numbers don't seem to correspond with the ones they slap on in the <usemask>, so they're probably using secondary referring ID. ARGH. So annoying. I mean, it would be easy enough to discern which filter is which once you have the lists and can click on a sample URL from each group to determine by eye which is which.


2005-08-05 04:48 pm (UTC) (Link)

Yeah, that's what I pretty much figured I'd have to do, after poking around with it some more!

I'm downloading the comma-separated version at the moment - I think I can pour it into Access and do what I need that way -- but if I can't, I would dearly love your help. Thanks for getting me this far!


2005-08-05 04:55 pm (UTC) (Link)

Yes, Access is better than Excel for this sort of thing.

Oh, if I had a dollar for every time I've tried to educate people on this. Excel is NOT a real database program, people. It's a SPREADSHEET application. Not the same thing.

Here's a sample I whipped up with my own data: (and obviously, you won't see anything useful if you click any of the links, so there, bawk, flap, Cluck!).

I just got the human filter names by clicking on the first link in each group to see which filter box was checked in the edit form, and then went and edited the HTML a bit. That, you'd obviously have to do yourself, even if I ended up helping.


2005-08-05 04:59 pm (UTC) (Link)

Excel drives me bugshit; if there's something I can do in both, I'll choose Access almost every time, unless the thing needed is actually a spreadsheet (like my monthly reports for the warehouse I worked at were). Assigning calculations to cells (and staring at them to see where I did it wrong) makes my brain ache; sorting, querying and categorizing, on the other hand. makes me a happy camper.


2005-08-05 05:03 pm (UTC) (Link)

Well, I actually hate Access because it's so unintuitive, so much harder to learn and use compared to FileMaker or MySQL, but it is a hell of a lot more powerful than Excel, and ignorant people misuse Excel all the time, which drives me CRAZY.

I actually think you will have no trouble with this!

I do wonder what happens if you have an old entry that uses a filter you have since deleted. Hmmm.


2005-08-05 05:11 pm (UTC) (Link)

I think it comes up as usemask, with the filter number being 0 (aka no existing filters specified) which, LJ says, makes it act like it's marked private.


2005-08-05 05:28 pm (UTC) (Link)

OK. Love the says. We'll see if that really pans out, huh? ;-)

Excel is not a real database program


2005-08-05 05:18 pm (UTC) (Link)


Sorry, just had to give an *amen* to that statement. Using Excel for a database is a big pet peeve of mine. you man now carry on about your normal business...

Re: Excel is not a real database program


2005-08-05 05:29 pm (UTC) (Link)

Fault partly goes to Microsoft for not bundling Access in with the other "Big Three" apps when they ship Office.


2005-08-05 11:04 pm (UTC) (Link)

It's a SPREADSHEET application. Not the same thing.

I'm sorry, we don't know each other, but I'd like to lick you now.


2005-08-06 12:24 am (UTC) (Link)

* Heee! * You are so silly. :-) You don't know me, but I've admired you from afar. I loved your Big Damn Essay, for example.


2005-08-05 04:30 pm (UTC) (Link)

Actually, huh, you could do all the filtering and sorting yourself, even in Excel, since it is comma-delimited.

Oh, cool.

Let me know either way.


2005-08-05 04:22 pm (UTC) (Link)

asdfhgfhg. I forgot about the asinine "You can only download a month at a time" thing.


2005-08-05 05:35 pm (UTC) (Link)

Actually, you could also use this opportunity to use TAGS to mark these entries for easier future access.


2005-08-05 06:17 pm (UTC) (Link)

Yup; that's one of the things I'm planning to do!