My friend threw together an app that scrapes your MySpace contacts and puts useful info into a reusable format.
DOWNLOAD IT HERE. (ZIP FILE)
UPDATE: It’s also available as a Torrent via The Pirate Bay. Please consider seeding this. It’s a tiny, tiny file.
Here’s the Read Me info I just put together to go with it:
“LOGIN_EMAIL”
and
“PASSWORD”
and change those.
LEAVE THE QUOTES IN PLACE
Save the file.
Upload these two files to your server.
point your web browser to http://where-you-put-the-file-on-your-server/ms_test.php
and what will result is a CSV file of all your MySpace friends and their demographic information. Also included is the URLs to “send message” etc, and some other useful things.
View the source of the page and copy it into a PlainText text file
Name the text file with the extension .csv
Now you should be able to work with your myspace friends in Excel
There is nothing malicious about this simple application. No viruses, spyware etc. It only does what it’s supposed to do: scrape your friends so you can more easily work with your social network data.
If you are of the camp that feels that people scraping their own myspace contacts is unethical, I suggest that you consider that all the pages are already available and the data they contain is rendered in HTML which can be freely accessed already. This is just a tool to make it easier to get the useful data separated from the clutter.
Finally, this is possibly against MySpace’s Terms Of Service, so use at your own risk.
Semantic Web Gang: Introductory Episode
Some suggestions:
1. During the conference call, use some sort of mixing program so the moderator can see who is talking at every moment during the recording via an audio level meter and make adjustments as needed.
2. Whenever some body new starts talking, quickly talk over them stating their name (it only obscures what they’re saying for one or two syllables so it’s easy for the listener to understand what they’re saying while taking in the metadata too)
3. Have a rotating or otherwise changing schedule of guests like TWiT does. The occasional random apperence by a CEO or two, or other dignitaries of the Web might help to keep the discussion interesting.
4. Don’t be afraid to spend a few hundred pounds on a decent microphone and maybe a mixer or or whatever is needed to improve the quality of the audio. The audio of Talking With Talis has been piss poor since the beginning. It would really serve you guys to improve on that.
I think part of your mission is evangelism, so I hate to think you’re losing audience because of the poor audio quality.
Looking forward to more!
-A
We started with the semantics of document structure. That’s what the World Wide Web is made of. It’s a giant network of HTML pages linking to each other. HTML (Hyper-Text Markup Language) documents have titles, links, headings and other elements that allow us to see web pages the way we do today. The whole idea of a “Hyper-Text” is referring to the power of a form of semantics. It is a matter of semantics that we see <a>this</a> as a link and
<h2>
</h2>
as a heading. It is the semantics of document structure a.k.a, HTML that have made it possible for documents like this one to link to others and for all of these pages that make up the Web to be rendered by our computers in more or less the same way.
The Idea of “The Semantic Web” is really only necessary for the sake of comparison.
So to sort out the semantics of what we’re talking about when we use the word “semantic” with regard to the Web, The Semantic Web refers to a movement toward not just semantics that define the structure of documents or pages, but semantics being applied to how information is made available over the Net.
Recent trends in the Web’s growth are making computer-language standards for compartmentalizing domains of data. The Semantic Web is a movement toward not just using semantics for defining document structure, but using semantics to make declarations about the context in which a linked resource or bit of information can be useful.
Maybe it’s a bit silly to say that there’s “a Killer App (as in one)” for Semantics. Nonetheless, Yahoo! announcing its search results will soon be taking advantage of Semantic Web Standards is definitely great news. Quote from the Yahoo! Search Blog:
“In the coming weeks, we’ll be releasing more detailed specifications that will describe our support of semantic web standards. Initially, we plan to support a number of microformats, including hCard, hCalendar, hReview, hAtom, and XFN. Yahoo! Search will work with the web community to evolve the vocabulary framework for embedding structured data. For starters, we plan to support vocabulary components from Dublin Core, Creative Commons, FOAF, GeoRSS, MediaRSS, and others based on feedback. And, we will support RDFa and eRDF markup to embed these into existing HTML pages. Finally, we are announcing support for the OpenSearch specification, with extensions for structured queries to deep web data sources.”
Hmmm. Wasn’t I just saying something about Semantics having an effect on Search Results in the near future? I guess Yahoo! doesn’t think that’s such a crazy idea.
Imagine that. Standards for defining the context in which information can be used actually being used to help search engines provide users with more relevant results. What a concept!
And on the SEO and SEM front, can you guess what Google, AOL, MSN and all the others are probably working on right now?
Big Rant.
Using HTML was once a smart move for findability online. Seems obvious to us now, but in case you don’t realize how stupid people were during the initial growth of the Web back in the late nineties, imagine this: People used to send cease and desist or take-down letters to owners of other sites because the other sites were linking to them.
“How dare you link to my site! You have no right to mention my existence and if you do not remove the link, I will sue you!”
In other words, we have a hard time looking beyond the current paradigm. Right now that paradigm is something like, in order to be findable, spend a lot of time working with the wording of your site’s copy, and make sure your metadata and you document structure are written to reflect what search results you want to win.
It’s funny though: Still, one of the best things you can do SEO wise is to have an RSS feed. And in case you didn’t realize this, RSS is a Semantic Standard. Apparently RSS 2.0 is a little convoluted (the adjustments made to the standard since it’s creation are not entirely in line with the Semantic Web school), but the original RSS stood for RDF Site Summary. Blah blah blah. Go look it up.
A little bit of Semantics is potentially way better for your site’s visibility than a whole lot Keyword tweaking.
FOAF, SIOC and the countless other Semantic Markups are a way for you to get your foot in the door now! A bit like the people that realized early on that they needed to have a website at all in the first place.
A little bit of early adoption of Semantics for your information could really pay off as we start moving toward a smarter Web. And we are moving toward a smarter Web. Who will be part of it when it reaches it’s tipping point for large scale adoption? Will you or your business? Or will you wait until some news report announces that the rest of the world has already gone semantic? I know I’ll be there. I already am.
Because what Search Engines are trying to do is provide users with access to what users are looking for, the process of SEO, when it consists of tweaking Keywords and/or document structure around, according to whatever the latest rumors are on what silly and temporary way Google seems to be currently making decisions about relevance, is always going to be flawed and as long as these SEO rumors are floating around, people will be trying to game the engines and in turn, people are collectively increasing the need for the engines to change their parameters, repeat, repeat, repeat. Search engines do not try to index sites based on the sites’ application of SEO techniques, engines index sites based on an attempt at creating an Information Architecture… This is hard to do because most website aren’t presented in a architecture-y way. So we’ve come full circle. Feeds are an architecture-y way to present your content, so it’s no wonder they help with SEO.
You might ask “So what’s next beyond RSS? How can I make Google love me even more?”
My answer is: “Stop lying to them with your SEO, and start helping them with Semantics”
And just remember what happened when a little bit of semantics got put into effect? Remember RSS? Well the blogosphere basically happened and in turn the “Live Web,” Podcasting and all that. Powerful stuff, and Web 2 is just the tip of the iceburg.
This post is aimed at one of my personal heroes, Professor Lawrence Lessig.
Mr Lessig,
First, I want to thank you for all the work you’ve done already to spread awareness about ‘Net Neutrality,’ the need for Intellectual Property reform, ‘Free Culture’ and so on. Your name comes up often as I do my part to help to change the way people think about the ownership of ideas and/or culture, no doubt because many of my thoughts on these matters are derivatives of yours. And finally, as an artist, thank you for helping me to see past my own possessive instincts, and to understand that my creative efforts are best honored if I aim for my work to become part of the Public Domain, because it is there that I can really contribute to the shape of our culture in the future. So thank you. Please keep up the good work.
It occurred to me that you may be the perfect person to spearhead the solving of a problem our government has -a small problem with major consequences. Before I go on though, I just want to urge you not to take this letter the wrong way. I don’t mean to imply that you need people like me to help you to choose your battles. But I know of no one else in the public eye that is such an advocate for the people, and who also seems to understand the implications of digital communication via the Web. You are the only public figure I can think of that generally seems to take the people’s side in all the domains where this issue manifests itself: The need for transparency in government; The need for people to be able to navigate the law to some degree without the aid of lawyers; The importance and potential of the [Read/Write] Web, especially with regard to how it can and does make our Democracy more democratic; etc… You actually seem to understand what the Web is and why it is important, and I fear that many or most of our legislators, judges and executives do not. This is why I’m writing to you.
The problem is that government websites generally lack consistency, search-ability, interactivity and general user-friendliness. On the surface, this may seem to many people like a minor problem. But from my point of view, it is one of the most important manifestations of how our government doesn’t work for the average person. This is a huge opportunity to improve how our democracy works for us.
Here are some of my thoughts on this.
1. Government websites generally have no interoperability between them. It seems to me that government websites should share a common information infrastructure as well as a common basic user interface and query system. If I am looking for information on something like a law on one government site, like say a county, I should be able to expand my search to include less local results, like say the state I am in, or narrow my search to only include more local results, like the City I am in. I think that government sites should be hierachically connected wherever possible to say the very least.
In general, I think it is time for all official government agency websites to become integrated.
2. Government websites do not routinely take advantage of technologies that make it easy for us to get new information from them. With technologies like RSS and iCal, it seems that citizens should be able to access regular updates from all the government agencies that concern them. We should be able to anonymously subscribe to feeds of governmental news, events, changes in policy, Etc. Example: “Effective today: All automobiles must have headlights turned on when it is raining regardless of the time of day. See Vehicle Code XYZ Section abc.”
3. By allowing existing laws to be un-findable, our government excludes us from even being able to understand what we have supposedly agreed upon through a democratic process.
For instance, on many occasions, I have tried to find out the specifics of one law or another. I have gone to my City, County and State government websites hoping for my question to be answered by a quick search, but instead, I’ve found myself hours later with a ton of windows open still trying to figure out the answer to a specific question like “Is [somehting] against the law?”
Again, to some people this may seem like a trivial complaint, but how in the world are we supposed to be law-abiding citizens if we cannot even be sure what the laws are? I believe that most people, in most communities in the USA have a very vague understanding of what is and isn’t legal. To many of us, The Law acts like some sort of urban mythology. We have no idea what the law actually says, and we cannot find the law if we want to learn what it actually says.
I have even had conversations with law enforcement officers in which the officers assured me that I “Can’t do” something, but were unable to tell me what the law says, where it says it, whether it is a local, state or federal law that is in question, or where I could even begin to look to find out for myself. This is scary to me.
I understand that Laws themselves are often confusing to lay persons. But I don’t understand why it is so hard to even find Laws in the first place. We have the technology to vastly improve this situation. It must be improved.
4 . Government websites generally have no place for public discussion or comment. There is also generally no universal protocol for asking the government(s) questions through the Web. Really, there is practically no way to reliably get facts about policy from government agencies in general. Since we clearly have the technology to make it possible for citizens to interact with and get information from government agencies, while keeping the expense to taxpayers very low, shouldn’t this be imperative?
So those are some of my main ideas about the digital government interface. Perhaps it is time for it to become written into law that certain standards and improvements are implemented on all government websites. Indeed, if there are already legally binding standards in place for government websites, they need to be vastly improved.
If technologies like RSS along with Semantic Web technologies were taken advantage of by government agencies, they could lead to vast improvements in our ability to understand and take part in our democracy.
Mr. Lessig, I wanted to write this to you because I don’t know where else to turn with these ideas. I hope you get this, and if you do, I hope you understand why I wrote this to you, rather than, say, The President or Santa Clause.
Of course, I am more than willing to help with this cause in any way that I can.
Sincerely,
Andrew A. Peterson
Some steps I’m taking to get all my contacts in order:
SIOC (pronounced “shock”) stands for Semantically Interlinked Online Communities. It’s the emerging standard for distributing forum and blog discussions, if I’m understanding it right. Considering the possibilities of this, I think of trackbacks on steroids.
Here’s the example of SIOC from the Wikipedia
I think the C in SIOC is a little misleading.
Is there a way to download MySpace Friends? Not officially.I used “O-Community” to scrape my friends out of myspace and I used TextWrangler to turn the file into a series of <a> tags within <li> tags that include the XFN rel=”friend” attribute. I will be looking into how I might publish this information in a more relevant structure/format, probably as SIOC or FOAF data, if I can figure out how to do that.
I wish FaceBook & MySpace allowed you to download a spreadsheet (excel or whatever) of all your contacts and their demographic information like Age, Gender, Location Etc. Some other formats I would like to be able to download my contacts in are RDF/XML, vCard, CSV Etc…
Actually, it is explicitly against MySpace’s and FaceBook’s Terms Of Service to do what I have done here. Scripts and Bots are prohibited. Perhaps this will change some bright beautiful day in the future.
We need to be able to mine the ‘Data of Our Lives,’ including of course the contacts we make through Social Software services. We also need to be able to Back-Up all our Data.
If you would like help scraping your MySpace friends, drop me a line and maybe I can help out.
I love these Semantic Web diagrams and flow-charts that folks keep drawing up. They help me to stay excited.
This one is from Michael K. Bergman’s post, “Linked Data Comes of Age.”
Sorry for the shallow post. I do love the chart though. I mean by all means go read the article… Cool stuff is happening. There’s going to be the first ever Linked Data Planet Expo in NYC among other things mentioned there. Feels like momentum!
…and I like the pictures.
Loading...
2012 andrewapeterson.com. Proudly Powered by WordPress. Design Based on the "Twelve" Theme.