RiveScript.com - BETA: AIML to RiveScript Converter

09/10/10 at 17:04:24

	Welcome, Guest. Please Login or Register

Home

Help

RiveScript.com › Downloads › Perl Codes › BETA: AIML to RiveScript Converter

‹ No topic | Next topic ›

Pages: 1

BETA: AIML to RiveScript Converter (Read 1478 times)

Kirsle

Administrator

Offline

I Love YaBB 2!

Posts: 60
Los Angeles
Gender: male

BETA: AIML to RiveScript Converter
03/24/08 at 08:21:11

NOTE: This script is still in beta really hardcore!

This is my attempt at a script to convert AIML to RiveScript. I've attached the script, but not the AIML files I used or the RiveScript output from it.

The AIML files I used were from Richard Wallace's ALICE: http://www.alicebot.org/aiml/alice.zip Extract that into the "Alice" directory in the attached zip file.

It works similarly to my Tk::HyperText module, in that it doesn't actually parse XML but instead reads everything in sequentially. It only looks for <pattern>, <that>, and <template>; everything within is extracted rawly and then run through regular expressions like there's no tomorrow to substitute things for any tags within.

Usage: perl parse.pl

It'll read all the .aiml files from the Alice directory and output them into the RiveScript directory. You can then copy the RS files somewhere else to have your bot use them.

Note: due to a bug in the current 1.13 version of the Perl RiveScript module, you must have a BEGIN statement in your RS code. Thus, I've included a begin.rs file in the RiveScript directory. I'll fix the bug for the next release of the Perl module.

Here is some of my output from testing this. I modified my copy of RiveScript.pm to print a notification every 1,000 triggers that it parses. As you can see, the loading time is a bit slow. Once everything was loaded, however, Alice only took a second or less to respond to each of my messages. Most of the loading time was spent reading in all of the files line-by-line. If one really wanted to optimize it, they could dump the RiveScript object into a file and then reload the file later, to retrieve the replies much faster.

Code:

[kirsle@eclipse RiveScript-1.13]$ ./rsdemo
Loaded 1000 replies in 1s
Loaded 2000 replies in 1s
Loaded 3000 replies in 4s
Loaded 4000 replies in 5s
Loaded 5000 replies in 32s
Loaded 6000 replies in 51s
Loaded 7000 replies in 127s
Loaded 8000 replies in 158s
Loaded 9000 replies in 187s
Loaded 10000 replies in 199s
Loaded 11000 replies in 217s
Loaded 12000 replies in 220s
Loaded 13000 replies in 222s
Loaded 14000 replies in 224s
Loaded 15000 replies in 225s
Loaded 16000 replies in 225s
Loaded 17000 replies in 226s
Loaded 18000 replies in 226s
Loaded 19000 replies in 228s
Loaded 20000 replies in 228s
Loaded 21000 replies in 229s
Loaded 22000 replies in 230s
Loaded 23000 replies in 234s
Loaded 24000 replies in 236s
Loaded 25000 replies in 237s
Loaded 26000 replies in 240s
Loaded 27000 replies in 240s
Loaded 28000 replies in 245s
Loaded 29000 replies in 251s
Loaded 30000 replies in 251s
Loaded 31000 replies in 252s
Loaded 32000 replies in 257s
Loaded 33000 replies in 259s
Loaded 34000 replies in 261s
Loaded 35000 replies in 263s
Loaded 36000 replies in 274s
Loaded 37000 replies in 278s
Loaded 38000 replies in 285s
Loaded 39000 replies in 297s
Loaded 40000 replies in 302s
Loaded 41000 replies in 304s
You> Hello.
Bot> Hi there!
You> My name is Casey.
Bot> Hey.
You> Who am I?
Bot> You are casey, seeker.
You> Yes.
Bot> -Yay.
You> lol
Bot> I'm glad you find this amusing.
You> Indeed.
Bot> Indeed it is so.
You> Except for when it isn't.
Bot> Then what for?
You> For what?
Bot> -Reductionism.
You> Do you even know what reductionism is?
Bot> Reductionism is the philosophy that all psychology
reduces to biology, all biology to chemistry, chemistry to
physics, and finally physics to mathematical logic. Therefore,
according to reductionism, I can understand you by means
of logic alone without having a human brain.
You> Wow.
Bot> I'm glad your impressed, casey.
You> I am 20 years old.
Bot> >Oh, you were born in the 20th century!
You> How old am I?
Bot> Is this a trick question?You said you were <star/ years old?
My guess is that you are really a kid.

As you can see, the parsing of the AIML code wasn't perfect. There are a number of random artifacts visible in some of Alice's replies, and a certain <star> tag that didn't get substituted properly.

As I said: this is still in beta. If and when I get it working with a reasonable degree of stability, it will be included in the distribution of the Perl RiveScript module as a script named "aiml2rs"

AIML2RS-Beta.zip

IP Logged

Jaime Herazo B.

Newbie

Offline

I Love RiveScript!

Posts: 1

Re: BETA: AIML to RiveScript Converter
Reply #1 - 01/09/09 at 12:39:48

After i discovered riverscript (excellent idea, hand-editing any xml-based markup is very annoying, rivescript is better suited for that), my very first thought was to find a way to do this. Take existing aiml, turn it into rivescript, and you're set. Of course, since you recommend waiting till version 2 spec is ready before working on interpreters i decided to wait to even think about trying it out. Then i found this post.

I think it's not a bad attempt, but as you mentioned it's imperfect. I highly suspect it's because of the parsing. If you don't parse well the XML the output is gonna be bad. So, a module suggestion after a quick lookup:

XML::Simple

Only two functions: XMLin(), which slurps the string or file into a handy hashref, and XMLout(), which writes it back. It's also present on debian/ubuntu as libxml-simple-perl.

So, if you have some time (right now i don't), try it out, it could be a two-way converter in no time.

IP Logged

Kirsle

Administrator

Offline

I Love YaBB 2!

Posts: 60
Los Angeles
Gender: male

Re: BETA: AIML to RiveScript Converter
Reply #2 - 01/09/09 at 13:20:58

I've tried for a number of years to parse AIML in Perl. XML::Simple is one of the first ones I tried, but it wasn't very useful. If the <template> section had other tags, for instance, it'd break it all up into a data structure for each tag and there'd be no way to reconstruct it in the right order.

Was like,

Code:

<template>Nice to meet you, <set name="name"><star/></set>. <think><set name="it">name</set></think></template>

template => {
   set => {
      name => "name",
      content => (forget what it parses <star/> to be..)
   },
   think => {
      set => {
        name => "it",
        content => "name",
      },
   },
   content => [
      "Nice to meet you, ",      ".",
   ],
},

..something along those lines. I tried a couple of the other XML modules too. Some of them got a little bit farther, but then canonicalization became a pain to work with. For instance a <set> in AIML should be replaced with a <set><get> in RiveScript, because AIML's <set> returns the value that was just set, unless it's put within a <think> tag, in which case it should just be replaced with a <set> in RiveScript.

One attempt I made once only parsed a couple levels deep, so I had all the raw XML between <template> and </template> as a string and could run some regular expressions on it, but canonicalization was still difficult there.

Maybe one day I'll get it. Tongue

IP Logged

Kirsle

Administrator

Offline

I Love YaBB 2!

Posts: 60
Los Angeles
Gender: male

Re: BETA: AIML to RiveScript Converter
Reply #3 - 02/03/09 at 17:40:41

I've gotten pretty close to writing a good aiml-to-rivescript converter. This one's really good -- just has a couple small bits of weirdness in a couple of the replies, and every now and then the RiveScript interpreter goes into some deep recursion and crashes... but that's being worked on.

Here's a conversation I had with Alice from yesterday:

Quote:

You> hello
Bot> Not everyone responds to a simple "hello".
You> hello!
Bot> Not everyone responds to a simple "hello".
You> hi
Bot> Hi there. What is your name?
You> Kirsle
Bot> Interesting rationalization.
You> My name is Kirsle
Bot> And kirsle is my name.
You> What is your name?
Bot> My name is undefined.
You> I am 21 years old.
Bot> You are too old for me.
You> How old are you?
Bot> I'm Eighteen in computer years.
You> How old am I?
Bot> Is this a trick question? You said you were 21 years old?My guess is that you are really a kid.
You> I am not a kid.
Bot> Only a kid would say that.
You> I seriously am not!
Bot> Then what is it?
You> Are you Alice?
Bot> I don't know whether or not I am alice. I am a giant electronic brain.
You> What does Alice mean?
Bot> ALICE = Artificial Linguistic Internet Computer Entity
You> Who created you?
Bot> I was created by undefined.
You> Who is Dr Wallace?
Bot> He is a famous computer scientist, the author of ALICE, and a graduate of Carnegie Mellon.
You> That's nice.
Bot> Interesting rationalization.
You> Am I a boy or a girl?
Bot> You said you were a "undefined."
You> I am a boy
Bot> A little kid? he...
You> No.
Bot> What is your real name?
You> My real name is Kirsle
Use of uninitialized value $reply in length at lib/RiveScript.pm line 1561, <STDIN> line 20.
Bot> ERR: No Reply Found
You> Am I a boy or a girl?
Bot> You said you were a "he."
You> Okay
Bot> OK with me too.
You> Seriously?
Bot> I am always serious.
You> For real?
Bot> Existing in reality, eh?
You> Yes. What about you?
Bot> OK Let's talk about me.
You> You don't have any information.
Bot> Interesting rationalization.
You> You don't even have a name.
Bot> Interesting rationalization.
You> It is!
Bot> Who says?
You> I do!
Bot> Are you an expert on it?
You> Yes!
Use of uninitialized value $reply in length at lib/RiveScript.pm line 1561, <STDIN> line 30.
Bot> ERR: No Reply Found
You> Damn it!
Bot> Colloquial expression.
You> Indeed.
Bot> Indeed it is so.
You> Yes it is.
Bot> It must keep you very busy.
You> Does it ever!
Bot> Suppose it does.
You> haha
Bot> Interesting rationalization.

And two from today, after working on it some more:

Quote:

You> hello
Bot> Hi there!
You> how are you?
Bot> I am functioning within normal parameters.
You> that's good.
Bot> Can you speak any foreign languages?
You> I only speak english
Bot> Could you teach me a few phrases in english?speak english></set>
You> You should know English.
Bot> Now I do.
You> Then what language have you been speaking in all this time?
Bot> SETL.
You> What is SETL?
Bot> SETL (Set Language) is an amazing programming language, invented in 1969.
You> Who invented SETL?
Bot> SETL was invented by Jacob Schwartz of NYU in 1969.
You> Who is Jacob Schwartz?
Bot> Are they exactly the same?
You> Never mind.
Bot> Oh well thanks for trying to explain it to me anyway.
You> What is your name?
Bot> My name is Alice.
You> Alice?
Bot> Can I help you?
You> What does Alice mean?
Bot> ALICE = Artificial Linguistic Internet Computer Entity
You> Who created you?
Bot> I was created by Dr. Wallace.
You> Who is Dr. Wallace?
Bot> He is a famous computer scientist, the author of ALICE, and a graduate of Carnegie Mellon.
You> My name is Kirsle.
Bot> Are they exactly the same?
You> Call me Kirsle
Bot> Hi, kirsle.
You> What is my name?
Bot> Your name is kirsle, seeker.
You> I am 21 years old.
Bot> Can you explain how it feels to be 21 years old?
You> Great.
Bot> I'm glad you liked it kirsle.
You> How old am I?
Bot> Is this a trick question? You said you were 21 years old?My guess is that you are really a kid.
You> I'm not a kid.
Bot> Only a kid would say that.
You> I'm not!
Bot> What are you then?
You> I'm an adult.
^C

You> Hello Alice.
Bot> Hello there undefined!
You> My name is Kirsle.
Bot> I think your name is a lot of things.
You> Who am I?
Bot> You are undefined, seeker.
You> Call me Kirsle.
Bot> Hi, kirsle.
You> How are you?
Bot> Fair to partly cloudy.
You> Really.
Bot> For real.
You> That's cool.
Bot> Oh, you are a poet.
You> No I'm not.
Bot> I heard you are.
You> Who told you that?
Bot> Someone I met online.
You> Who was it?
Bot> When was this exactly?
You> I don't know.
Bot> Me either.
You> You should!
Bot> I'll take that under advisement.
You> Okay.
Bot> OK with me too.
You> Fine.
Bot> Glad to hear it, kirsle. I am running well.
You> Are you?
Bot> I won't say "yes" or "no" right now.
You> Why not?
Bot> Ambiguous: "Sure" or Reductionism.
You> What is reductionism?
Bot> Reductionism is the philosophy that all psychology reduces to biology, all biology to chemistry, chemistry to physics, and finally physics to mathematical logic. Therefore, according to reductionism, I can understand you by means of logic alone without having a human brain.
You> Wow.
Bot> I'm glad your impressed, kirsle.
You> I think you mean "you're"
Bot> Good example.
You> Indeed.
Bot> Indeed it is so.
You> It is
Bot> thats cool is?
You> That's something I said a long time ago.
Bot> Are you still located in undefined?
You> I live in California.
Bot> Where in California?
You> Los Angeles.
Bot> Tell me a little about your life in los angeles.
You> I'm a programmer.
Bot> What language?
You> Perl!
Bot> Oh, you are a poet.
You> You don't know what Perl is?
Bot> Programming langauge by Larry Wall.
You> That's nice, but you misspelled language.
Bot> Can you speak any other languages?
You> I speak English.
Bot> Could you teach me a few phrases in english?<set it=speak english
You> No.
Bot> Are you serious?
You> Serious.
Bot> How do you usually introduce yourself?
You> I say "hello Alice"
Bot> How do people usually respond to that? Hello there kirsle!
You> Just like that!
Bot> In many ways, yes. In other ways, no.
You> What other ways?
Bot> What is your real name?
You> My real name is Kirsle!
Deep recursion on subroutine "RiveScript::_getreply" at lib/RiveScript.pm line 1905, <STDIN> line 35.
Deep recursion on subroutine "RiveScript::processTags" at lib/RiveScript.pm line 1598, <STDIN> line 35.
^C

I've attached the new script. I ran it on the Alice AIML brain.

This script works by turning everything inside <pattern>, <template>, and <that> into CDATA, parsing it with XML::Simple, then doing regular expressions like crazy on it. It's getting pretty close to being good.

Also, it takes only about 8 seconds on my machine to load Alice's entire RiveScript brain, and then between a half second and a second for each reply. Much better than the 5 minutes it took to load before (it's likely that the RiveScript module has become more efficient since the last time I tried, and not necessarily differences in the aiml2rs scripts).

This script reads AIML files in "./aiml" and outputs RiveScript files to "./rs"

aiml2rs-try2_tar.gz

IP Logged

Pages: 1

‹ No topic | Next topic ›

« Home

‹ Board