log in | register | forums
Show:
Go:
Forums
Username:

Password:

User accounts
Register new account
Forgot password
Forum stats
List of members
Search the forums

Advanced search
Recent discussions
- In praise of !ShareFS (News:9)
- Aemulor (Gen:9)
- RISC OS London Show 2017 (News:)
- Acorn-Gaming.org.uk (Games:9)
- Help getting RPCEmu working on a MacBook (Gen:10)
- Best BBC Micro game music (Games:42)
- !DualHead second release reviewed (News:)
- Equinox (Games:2)
- RiscStation 7500 Scorcher needs a home (Gen:4)
- September News round-up (News:)
Latest postings RSS Feeds
RSS 2.0 | 1.0 | 0.9
Atom 0.3
Misc RDF | CDF
Site Search
 
Article archives
The Icon Bar: The Playpen: Poogle
 
  Poogle
  This is a long thread. Click here to view the threaded list.
 
Jeffrey Lee Message #80187, posted by Phlamethrower at 20:57, 14/9/2006
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
...would be a great name for a software search engine. Like, that one that I invented the idea for years ago and then didn't get very far with producing it. And what with the beginnings of some sensible cooperation between different RO-related projects, and my recent (still unreleased) auto packager uploader thingy, it may be worth revisiting the idea.

In summary:
  1. We need an XML format for describing software packages
  2. We need a spider bot to trawl the internet for the XML files
  3. We need a search engine to allow people to go through the results
  4. Also any other bells and whistles, e.g. RSS feeds for software updates, some form of RiscPkg integration/compatibility/cooperation
  5. Reveal my creation to the world and allow me to take my rightful place as the king of the Internet
Points 2 and 3 are easy, there's plenty of software we can borrow code/algorithms/techniques from. The big issue is point 1, getting the XML format right.

So, do we have any XML gurus in the house? Has anyone seen any noteworthy attempts by other people to develop a similar system? (So that we may assasinate them, steal their ideas, or work with them, in my order of preference)
  ^[ Log in to reply ]
 
Jason Tribbeck Message #80206, posted by tribbles at 22:34, 14/9/2006, in reply to message #80187
tribbles
Captain Helix

Posts: 929
I'm an XML guru (have been doing XML for a heck of a lot of time). I was part of the WAP Forum for a bit, and got a couple of attributes added to the UAProf standard.

Also got the XSL/DTD RISC OS filetype numbers allocated.

Haven't defined a format for software packages, but have done the spidering/searching side of things a lot.

I'd imagine there are some other standards out there for other platforms - it's not something I've needed to look for. If there's no XML format, then it shouldn't be too difficult to design. All you need to do really is know what you want to keep.
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80208, posted by Phlamethrower at 22:41, 14/9/2006, in reply to message #80206
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
Yay!

All you need to do really is know what you want to keep.
I think that's the problem :P
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80211, posted by Phlamethrower at 22:50, 14/9/2006, in reply to message #80208
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
For a start, the icon would be a poodle. Mainly because Poogle sounds like poodle, and poodles have a habit of making poo.
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80212, posted by Phlamethrower at 23:02, 14/9/2006, in reply to message #80211
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
Open Software Description? It looks a bit basic for our requirements (we'd need stuff for descriptions of the software, author, home page, etc.), but it's a good start (even if it doesn't have RISC OS as a valid OS or ARM as a valid CPU!). It also has dependencies between packages listed.

Although their examples don't show it, you can specify a SOFTPKG instead of a CODEBASE as a dependency. That way you can use the unique name of the package, thus allowing the system to identify a download location even if the original website of the dependent package died.
  ^[ Log in to reply ]
 
Jason Tribbeck Message #80213, posted by tribbles at 23:03, 14/9/2006, in reply to message #80208
tribbles
Captain Helix

Posts: 929
Okay - at a first glance, what I suggest is as follows (tags could be subject to change):
<product>
<name>Name of product</name>
<id>Unique ID (assigned by 3rd party)</id>
<author>Software author 1</author>
<author>Software author 2</author>
<producer>Software producer</producer>
<documentation>Documentation URL</documentation>
<homepage>Homepage</homepage>
<version number="current-version">
<changelog>Changelog URL</changelog>
<download>Download URL</download>
<compatible name="os-name1" version="os-ver1"/>
<compatible name="os-name2" version="os-ver2"/>
<prerequisite id="prereqid1" />
<prerequisite id="prereqid2" />
</version>
<version number="older-version">
<changelog>Changelog URL</changelog>
<download>Download URL</download>
<compatible name="os-name1" version="os-ver1"/>
<compatible name="os-name2" version="os-ver2"/>
<prerequisite id="prereqid1" />
<prerequisite id="prereqid2" />
<version>
</product>

The IDs are the thing that tie everything together. I've also put two of everything which you could have more than 1.

Note that this is quite basic, and I'm sure that everyone will have their own ideas (you could use namespaces to separate the computer-related stuff to the stuff that humans are meant to look at.

The only thing is that I'm sure that there is something like this - didn't one of the RISC OS software search sites use XML?
  ^[ Log in to reply ]
 
Jason Tribbeck Message #80214, posted by tribbles at 23:05, 14/9/2006, in reply to message #80213
tribbles
Captain Helix

Posts: 929
Of course, I knew someone would find something while I was writing that :)

The OSD looks like a good starting point - bit old though.
  ^[ Log in to reply ]
 
Phil Mellor Message #80216, posted by monkeyson2 at 23:09, 14/9/2006, in reply to message #80214
monkeyson2Please don't let them make me be a monkey butler

Posts: 12380
Hasn't risc-os.net already done this?
  ^[ Log in to reply ]
 
VinceH Message #80218, posted by VincceH at 23:09, 14/9/2006, in reply to message #80187
VincceH
Lowering the tone since the dawn of time

Posts: 1583
6. You need somewhere to put the central registry.

The name riscository kind of works. The pun is actually (obviously) repository, but still...

If you get to that point, I believe I can set you up with a suitable subdirectory/subdomain and access rights on riscository.com if you'd like.

(He says, not actually knowing if the hosting facilities he has allow that sort of thing).
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80230, posted by Phlamethrower at 23:36, 14/9/2006, in reply to message #80218
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
If we go with the plan of having each XML file link to other XML files (just for the sake of linking to them), then you won't need a central repository. Each site that offers a search engine/RSS feed/whatever can spider its way through all the files and build its own registry.

Of course, that doesn't mean a central registry wouldn't be useful. But it would be good if the system was designed so that it didn't need one. Links to other XML files and a standard naming convention for hosting the XML files on websites (i.e. files should be called poogle.xml) should be all that's needed.

We'll probably host a registry/search script/RSS feed/whatever on TIB.

Hasn't risc-os.net already done this?
Dunno - their site seems to be down. And they obviously didn't advertise it well enough for me to hear about :P
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80239, posted by Phlamethrower at 01:00, 15/9/2006, in reply to message #80230
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
Note to self: Don't forget to read the bit on nutshells where it says using RDF+Dublin Core is a good idea :P

W3C's note about OSD says RDF is good too.

So now i'm having a nosey for an appropriate RDF grammar thing.

In fact, we may be able to use RSS as the format, since we'll be providing summaries of software available on websites. Then we can find/add the missing bits we need for RSS to describe software.

...which is essentially what was said on the nutshells site :P (Except making no direct mention of having everything done in RSS).
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80240, posted by Phlamethrower at 01:02, 15/9/2006, in reply to message #80239
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
Writing it in RSS would also be a good opportunity for me to write this. Poodle and !ARSS, a match made in heaven.
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80241, posted by Phlamethrower at 01:22, 15/9/2006, in reply to message #80240
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
DOAP? It sounds like it was spawned from similar feelings about keeping software repositories up to date:

So many registries now exist that keeping them up to date has become a real problem. The release cycle for diligent software maintainers often involves visits to several Web sites to keep the information up to date, not to mention updating their own Web sites. However, such maintainers are few and far between, and it's not uncommon to find out-of-date information in a registry. That this data gets out of date is unsurprising when you consider the aspects that many modern software projects involve: mailing lists, IRC channels, Web sites, wikis, CVS repositories, and so on.
Use cases for project descriptions include:
  • Easy importing of projects into software directories
  • Data exchange between software directories
  • ...
Of course this is designed for open source software, but I'm sure it can be applied to closed source as well.

Now, if only I had a program to convert something written in the RDF vocabulary description language thingy into plain english :P

[Edited by Phlamethrower at 01:23, 15/9/2006]
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80242, posted by Phlamethrower at 01:41, 15/9/2006, in reply to message #80241
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
DOAP looks like a decent way of describing software. It even has a way of specifying the old URL's of projects :)

Now we need a way of linking to other abritrary DOAP files.
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80243, posted by Phlamethrower at 01:50, 15/9/2006, in reply to message #80242
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
ROR looks like a good way of packaging the DOAP/whatever up onto a website. They've already standardised the use of ror.xml for provding a listing of whatever resources your website contains. Unfortunately they don't seem to have a "software" resource type.

But: http://www.rorweb.com/rorsuggest.htm
Even if it doesn't have a way for you to justify your suggestion :(
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80245, posted by Phlamethrower at 01:58, 15/9/2006, in reply to message #80243
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
Gah! Why does everything have to alll be so conceptual! There's probably an RDF vocubulary for describing RDF vocabularies that describe RDF vocabularies that are described using RDF vocabularies and use confusing acronyms to confuse the reader and obfuscate the purpose of the RDF vocabulary that is being described!
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80246, posted by Phlamethrower at 02:03, 15/9/2006, in reply to message #80245
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
I guess we could use the rdfs:seealso property to provide the list of other XML files. Or maybe a slight bastardisation of DocumentList where the document names are URL's (Unless I'm missing something where you can name a document and give a URL).

Help me obi-tribbeck, you're my only hope!
  ^[ Log in to reply ]
 
Jeffrey Lee Message #80252, posted by Phlamethrower at 09:38, 15/9/2006, in reply to message #80246
PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

Posts: 15055
Perhaps: A ROR containing DOAPs with a DocumentList at the end of the file.

Or we could just steal all their ideas and write our own vocabulary. Isn't there some way of specifying that one property maps onto the definiton of another? So we could just use all their definitions but restructure it into what suits us?
  ^[ Log in to reply ]
 
James Shaw Message #80288, posted by Hertzsprung at 19:11, 15/9/2006, in reply to message #80187
Hertzsprung
Ghost-like

Posts: 1746
...would be a great name for a software search engine. Like, that one that I invented the idea for years ago and then didn't get very far with producing it. And what with the beginnings of some sensible cooperation between different RO-related projects, and my recent (still unreleased) auto packager uploader thingy, it may be worth revisiting the idea.

In summary:
  1. We need an XML
Strike One!
format for describing software packages
  • We need a spider bot to trawl the internet for the XML
    Strike Two!
    files
  • We need a search engine to allow people to go through the results
  • Also any other bells and whistles, e.g. RSS feeds for software updates, some form of RiscPkg integration/compatibility/cooperation
  • Reveal my creation to the world and allow me to take my rightful place as the king of the InternetPoints 2 and 3 are easy, there's plenty of software we can borrow code/algorithms/techniques from. The big issue is point 1, getting the XML
    Steee-rike Three! You're out!
      ^[ Log in to reply ]
     
    Jeffrey Lee Message #80318, posted by Phlamethrower at 15:36, 16/9/2006, in reply to message #80288
    PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

    Posts: 15055
    Steee-rike Three! You're out!
    So, how would you suggest that we do it? :P

    risc-os.net is back up!

    It doesn't look very good :(
      ^[ Log in to reply ]
     
    Jeffrey Lee Message #80319, posted by Phlamethrower at 16:15, 16/9/2006, in reply to message #80318
    PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

    Posts: 15055
    Revised idea:

    * Skip ROR for now, it looks to be unnecessary
    * Instead have an RDF file containing DOAP and DocumentList entries
    * The DOAP entries would obviously describe the software
    * The DocumentList entries would obviously list other RDF files which the spider bot should look for
    * After looking at the DocumentList examples, it's clear that they can contain both a URL and a name.

    Yes/No?
      ^[ Log in to reply ]
     
    Jeffrey Lee Message #80320, posted by Phlamethrower at 16:19, 16/9/2006, in reply to message #80319
    PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

    Posts: 15055
    When a mirror site decides to mirror your software, it can produce a DOAP entry that's near identical to the original, but add doap:download-mirror to list the mirrored download, doap:screenshots for its own screenshot page, etc.

    Since the software name is meant to be unique, a search engine crawler could aggregate all the information from different mirror sites to produce one long list of mirrors for a particular piece of software (complete with indexing them by what version of the software each mirror provides)
      ^[ Log in to reply ]
     
    Jeffrey Lee Message #80321, posted by Phlamethrower at 16:23, 16/9/2006, in reply to message #80320
    PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

    Posts: 15055
    We could also suggest that people include a DOAP XML file in the .zip file for each piece of software. This would be useful for identifying the software either while it's still on the internet (e.g. if the crawler can't decide what version the software is by looking at referring DOAP entries), or once downloaded to someone's machine (so the user or some package management thingy can identify what the software is)
      ^[ Log in to reply ]
     
    James Shaw Message #80331, posted by Hertzsprung at 23:09, 16/9/2006, in reply to message #80318
    Hertzsprung
    Ghost-like

    Posts: 1746
    Steee-rike Three! You're out!
    So, how would you suggest that we do it? :P
    JSON, of course.
      ^[ Log in to reply ]
     
    Jason Tribbeck Message #80332, posted by tribbles at 23:12, 16/9/2006, in reply to message #80331
    tribbles
    Captain Helix

    Posts: 929
    JSON, of course.
    http://en.wikipedia.org/wiki/JSON

    How does that help?
      ^[ Log in to reply ]
     
    Phil Mellor Message #80333, posted by monkeyson2 at 23:17, 16/9/2006, in reply to message #80332
    monkeyson2Please don't let them make me be a monkey butler

    Posts: 12380
    Is that JSON TRBBCK?
      ^[ Log in to reply ]
     
    Jeffrey Lee Message #80510, posted by Phlamethrower at 00:41, 26/9/2006, in reply to message #80246
    PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

    Posts: 15055
    Help me obi-tribbeck, you're my only hope!
    Well?!?! :o
      ^[ Log in to reply ]
     
    Jason Tribbeck Message #80526, posted by tribbles at 09:18, 26/9/2006, in reply to message #80510
    tribbles
    Captain Helix

    Posts: 929
    Hey; I've been busy!
      ^[ Log in to reply ]
     
    Jeffrey Lee Message #80527, posted by Phlamethrower at 09:31, 26/9/2006, in reply to message #80526
    PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

    Posts: 15055
    That would have made for an interesting end to Star Wars :P
      ^[ Log in to reply ]
     
    Jeffrey Lee Message #81513, posted by Phlamethrower at 16:12, 20/10/2006, in reply to message #80510
    PhlamethrowerHot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot Hot stuff

    Posts: 15055
    Help me obi-tribbeck, you're my only hope!
    Well?!?! :o
    BIG BUMPING BENE GESSERITS
      ^[ Log in to reply ]
     
    Pages (2): 1 > >|

    The Icon Bar: The Playpen: Poogle