Bulbapedia Suggestion/offer: Use a database

[En-Cu-Kou]

(First time here – hopefully I'm doing everything right. Apparently the mods here have power to merge this into the appropriate thread if it doesn't deserve its own?)

There's a lot of inconsistencies on Bulbapedia. For example, as of now, Fury Cutter on Skorupi's Gen. IV learnset lists the accuracy as 90% (should be 95%). Outrage on Dratini's page has 10 PP (should be 15). Appeal numbers for Contests and Super Contests get mixed up at times. Aerodactyl's Gen. IV U-Turn even uses the wrong template line.
I don't think hundreds of such errors would be an underestimation.

A solution to this problem would be to generate these “boring” templates from a database (i.e., a place where each piece of information is stored exactly once). I've seen that at the start of the bot suggestions thread, Lin Zhen was talking about reating such a database... back in 2006. I don't think anything came out of that?

Anyway, I'll get to the point. Such a database is being assembled over at veekun, and it's freely available to anybody. I'm familiar with how it works, and would be glad to export files such as this one from it.

Of course, that database is not entirely complete (yet), or 100.00% correct. It is, however, consistent.

The missing link is a bot or script that would compare these generated templates to what's currently on Bulbapedia, and report any differences. These would come in three kinds:

• Template formatting differences (capitalization of links, leaving out optional arguments), which can be ignored or just fixed automatically
• Errors in the template generator (which are bound to happen – I'm not familiar with all of BP's conventions)
• Genuine errors, either in Bulbapedia or the database, which would have to be resolved and fixed in one of the places. (Hopefully this whole thing doesn't sound as “veekun leeching off BP”. I think BP would benefit a lot here; the database not so much.)

I'm not really familiar with Mediawiki bots, though I could, in time, give it a try if there's no better person for this.

...

Did that sound like it could be a good idea?

 

[kelvSYC]

I'd actually think that Bulbapedia would be better off with installing one of a number of Wikidata-related projects so as not to destroy its established MediaWiki base. For example: Semantic MediaWiki would facilitate keeping all of Bulbapedia's data in sync using constructs like:

in Magikarp (Pokémon): [[Learns::Tackle]]
in Tackle (move): [[Type::Normal]] [[Category:hysical]] [[Base Power::35]]

Then Semantic MediaWiki could be used to make a page that lists all Pokémon that learn a Normal-typed Physical move of 35 BP, including Magikarp, using its {{#ask}} query syntax. (Good for pages like Normal (type) or sections like the list of routes where a Pokémon is found. Stuff that has to be synchronized manually today.) The changes would be quite minor in nature, so as to not take down huge swaths of the wiki at a time.

I wouldn't be surprised if some of Bulbapedia's competitors such as pokemon.wikia.com already use Semantic MediaWiki (all Wikia sites can use Semantic MediaWiki on an admin's request).

Still doesn't address content issues like why Bulbapedia is still severely lacking in MD-specific stuff on the Pokémon's pages, but it's a start...

 

[MAGNEDETH]

not to de-rail your suggestion of an info database (boring templates?), but i would like to say that as a wiki, you are free to fix any errors you see.

granted, on the topic of an info database, it would be a bit the opposite of a community driven wiki.

 

[En-Cu-Kou]

I'd actually think that Bulbapedia would be better off with installing one of a number of Wikidata-related projects so as Inot to destroy its established MediaWiki base.

That would be extremely cool.
From looking at it, it would allow me to get answers to crazy questions like Which pokémon can learn 4 HM moves in generation 3?. Is that right? Makes me worried about the servers

not to de-rail your suggestion of an info database (boring templates?), but i would like to say that as a wiki, you are free to fix any errors you see.

A sisyphean task. As I mentioned there are hundreds of these, which is why I'm suggesting a semi-automated way to do this.

granted, on the topic of an info database, it would be a bit the opposite of a community driven wiki.

For the “boring” templates (level-up moves, infoboxes, etc.), I think the community would benefit from having the work cross-checked for consistency, if not automated entirely. Any differences would have to be resolved, by hand (unless there's an auto-editing bot, but I'm not making that). So if the community comes up with something creative and not boring, there'd just be a disagreement between the wiki and the database. No big deal. It's not like I want evil robots to take over Bulbapedia

 

[Zekurom]

granted, on the topic of an info database, it would be a bit the opposite of a community driven wiki.

A community-maintained info database, perhaps?

 

[kelvSYC]

As I said before, Bulbapedia admins could do this quite easily by installing the Semantic MediaWiki extension, and doing a series of massive edits in a huge community undertaking to make the existing data queryable. It's still a community-maintained info database (as I'd envision most of the semantic info to be inserted would be at the template level), and Semantic MediaWiki deployments have been used as such, especially in the health informatics field.

The benefits of Semantic MediaWiki is, for example, that a lot of manual syncing will be entirely replaced with wikicode. (Forget the type of Bite when you need to add it to a table? {{#show:Bite|?Type}} will do it for you!) Furthermore, you can also return results of a query as a list, a sortable table, parameters to a template, etc. (Now you can get a list of Pokémon who learn Thunderbolt by TM in alphabetical order, formatted in that electric yellow style!) The drawbacks is that adding a new "column" will still require you to edit 500+ pages (though having said that, if you know how to use Special:Ask, the generic query interface, you can get a list of, say, move pages for which their Mystery Dungeon Base Power is not known, so you know which articles to add that info to).

 

[immewnity]

Semantic MediaWiki isn't needed for that; we could simply have a template for that. Let me draft one up.

 

[Soulweaver]

I have a hunch Semantic MW would be more effective, though; templates aren't exactly light on server stress as far as I know.

 

[Werdnae]

The issue I see is that having it as a database on the front end won't fix the errors. If it's a publicly editable database, we'll still get errors. If it's not publicly editable, it goes against everything we stand for.

Also, I'm pretty sure that if Zhen Lin had wanted a Database rather than an encyclopedia, he'd have just done so when he started it. As it is, there's a database on the back end anyway. Mediawiki wasn't chosen because it was the easiest way for them to present the information, it was chosen because they wanted a publicly editable encyclopedia.

 

[Transfinite]

If it's a publicly editable database, we'll still get errors.

Wait, how is this different from any other page on the wiki?

 

[Zekurom]

Wait, how is this different from any other page on the wiki?

It isn't. I think what Werdnae was saying is that having errors is a trade-off for public editability.

 

[kelvSYC]

...and the thing about SMW is that we can have both public editability and a (pseudo-) DB frontend. SMW is just an extension of MW (born from the notion that structured data could and should be editable in wiki form so that it can be machine-processed, see Semantic MediaWiki and WikiData at meta); the fact is that the wiki principle remains the same. I agree that errors will still exist. But the highly structured nature of Pokémon data may just mean that errors are easier to control (maybe harder to fix, depending on the nature of the error, but if the accuracy of Bite on Blastoise's page is wrong, you'd look in Bite, not Blastoise, right?).

With SMW, there is no need for tradeoffs. If you want a wiki, you have one. If you want a DB, you have one, and it's the same as the wiki. Whether BP admins are up for it is another matter; there are some things that tend to be real stubborn round these parts...

 

[MisterE13]

Also, the basis of our status as a wiki shouldn't necessarily be in the movelists and stuff... After a certain point that should really never change until the release of a new game. Same for the power, accuracy, type, and damage category of Brine in Generation IV, if it's done right no-one will have to edit it anyways. We're as wiki because any user can rewrite the history of Dawn's Piplup in its entirety if they so choose to do so well, not because a user can spend a night making sure that the power of Earthquake is listed correctly in the article of every Pokémon who can use it.

 

[eric the espeon]

granted, on the topic of an info database, it would be a bit the opposite of a community driven wiki.

The issue I see is that having it as a database on the front end won't fix the errors. If it's a publicly editable database, we'll still get errors. If it's not publicly editable, it goes against everything we stand for.

Also, I'm pretty sure that if Zhen Lin had wanted a Database rather than an encyclopedia, he'd have just done so when he started it. As it is, there's a database on the back end anyway. Mediawiki wasn't chosen because it was the easiest way for them to present the information, it was chosen because they wanted a publicly editable encyclopedia.

As I understand it the database need not be mutually exclusive with community involvement. It will not remove all errors and close off editing forever, but flag up all the inconsistencies, all the pages which say something different from another page so that the community will be able to direct their efforts towards other projects rather than searching for these kind of errors and maintaining consistency over thousands of pages by hand. You can have a database in many forms using mediawiki, don't think of this as against the community but as aiding the community by allowing them to focus on things which need human thought rather than making them do work which can be largely automated (with the decisions still in the hands of humans).

Don't miss this chance!

 

[evkl]

Right, obviously the goal wasn't simply to get a database when we started it. That was one avenue that we decided to pursue. How hard would it be to present an example of the DB people are envisioning/proposing?

 

[kelvSYC]

Right, obviously the goal wasn't simply to get a database when we started it. That was one avenue that we decided to pursue. How hard would it be to present an example of the DB people are envisioning/proposing?

Examples of current SMW deployments include Familypedia, a genealogy wiki, SKYbrary, a wiki for flight information, and SNPedia, a wiki relating to bioinformatics. It's also used internally by organizations such as the US Department of Defense, Johnson & Johnson, and 3M.

 

[Misdre]

We (Poképédia from EP) can give some insights as well, since we have SMW installed for a long time (since the creation of the wiki I think). We don't use use it as much as we should but we will. It takes time to do it right - or how I want it to be.

(I'm running out of time right now, but I'll probably talk about the database vs. community-driven wiki debate, and then about SMW specifically, how it can be used, its performances and its problems)

 

[evkl]

Misdre, I'd love to hear more about your experiences with SMW and databases in general whenever you have a little more time.

 

[Archaic]

Likewise.

 

[En-Cu-Kou]

(sorry for the lag – time for my hobbies comes in bursts...)

This thread taught me more about what Bulbapedia is than reading the About page ever could. Thanks, guys

I think what Werdnae was saying is that having errors is a trade-off for public editability.

That doesn't have to be the case. I understand now that public editability has priority, but there should be a way to get consistency without sacrificing it.

Right, obviously the goal wasn't simply to get a database when we started it. That was one avenue that we decided to pursue. How hard would it be to present an example of the DB people are envisioning/proposing?

Umm... It's hard to give an example of a database… Basically it's a set of tables with a bunch of numbers and texts, as in this file, plus some information about what some of the numbers mean.
As you can see it's not too human-friendly in its raw form, but it's structured so that it's fairly easy to write a program to export data from it, for example in this form: (link). Same with infoboxes, lists, or whatever else.

IMO, this approach would be most in the spirit of Bulbapedia: setting up a page that would display some differences between BP and what the database “thinks” should be there, and letting people either fix BP or (tell someone to) fix the database or exporting program.

This way, anybody could edit normally, just like this project didn't exist – except their contributions would be checked against another resource.
And, Bulbapedia wouldn't even depend on the database – if the database or people who understand it go away, we'd be in the same situation we're in currently (minus errors that get fixed while it works).