| View previous topic :: View next topic |
| Author |
Message |
Albretch Mueller
Guest
|
 Posted: Thu Jun 19, 2008 8:03 pm Post subject: Mining Questions and Answers in Texts |
 |
|
I was wondering about the techniques and software that are used to achieve
this
Can you just get some plain text and feed it into some engine/parser and it
will give you as output (at least a good approximation of actual and
possible) questions in the text and if they are found in the text, their
corresponding answers?
Or, do you need to first parse/tag the whole text and go from there?
Thanks
lbrtchx |
|
| |
|
 Back to top |
Cameron Hughes
Guest
|
| Posted: Thu Jun 19, 2008 8:09 pm Post subject: Re: Mining Questions and Answers in Texts |
|
|
Albretch Mueller wrote:
| Quote: | I was wondering about the techniques and software that are used to achieve
this
Can you just get some plain text and feed it into some engine/parser and it
will give you as output (at least a good approximation of actual and
possible) questions in the text and if they are found in the text, their
corresponding answers?
Or, do you need to first parse/tag the whole text and go from there?
Thanks
lbrtchx
|
Without some sort of tagging of the original text, you're talking about
something that is non trivial.
While the '?' might or might not identify the questions in the text, knowing
where an answer starts
or stops (without tagging on some sort) requires ..... let's just say some
work :-)
Cheers
Cameron |
|
| |
|
| Back to top |
Albretch Mueller
Guest
|
| Posted: Thu Jun 19, 2008 8:46 pm Post subject: Re: Mining Questions and Answers in Texts |
|
|
Cameron Hughes wrote:
| Quote: | Without some sort of tagging of the original text, you're talking about
something that is non trivial. |
~
It sure isn't, but I think linguistics and NLP can help quite a bit here
and I wonder is there is some work already out there using what we have got
right now
~
As I see things the verb to be wears a big red clown nose
~
I am not a linguist but my hungry intuition tells me that there are ways
out there of building questions based on plain sentences
~
The meeting will be at 3 PM . . .
~
At what time will the meeting start?
~
I know this is very basic and naive I am just using it to clearly show what
I mean
~
| Quote: | While the '?' might or might not identify the questions in the text
~ |
Well actually I wasn't asking for a way to -identify- questions in a text,
but mining them
~
thanx
lbrtchx |
|
| |
|
| Back to top |
Cameron Hughes
Guest
|
| Posted: Fri Jun 20, 2008 5:46 pm Post subject: Re: Mining Questions and Answers in Texts |
|
|
Albretch Mueller wrote:
| Quote: | Cameron Hughes wrote:
Without some sort of tagging of the original text, you're talking about
something that is non trivial.
~
It sure isn't, but I think linguistics and NLP can help quite a bit here
and I wonder is there is some work already out there using what we have got
right now
~
As I see things the verb to be wears a big red clown nose
~
I am not a linguist but my hungry intuition tells me that there are ways
out there of building questions based on plain sentences
~
The meeting will be at 3 PM . . .
~
At what time will the meeting start?
~
I know this is very basic and naive I am just using it to clearly show what
I mean
~
While the '?' might or might not identify the questions in the text
~
Well actually I wasn't asking for a way to -identify- questions in a text,
but mining them
~
thanx
lbrtchx
|
If you're not asking about 'identifying questions in the text' then could
clarify what you
mean by 'mining questions in the text'
I'm sure you don't mean that you're trying to extrapolate what questions can
be asked
from some given text, because that could be infinitely many questions.
So please clarify... |
|
| |
|
| Back to top |
Albretch Mueller
Guest
|
| Posted: Sat Jun 21, 2008 4:37 am Post subject: Re: Mining Questions and Answers in Texts |
|
|
Cameron Hughes wrote:
| Quote: | If you're not asking about 'identifying questions in the text' then
could clarify what you
mean by 'mining questions in the text'
I'm sure you don't mean that you're trying to extrapolate what questions
can be asked
from some given text, because that could be infinitely many questions.
So please clarify...
~ |
Well, you helped me see my bias
~
The thing is that right now I am trying to extract questions and answers
from technical specifications and of course, since they are such
(specifications) you may just get all conjugations of the verb to be and do
the parsing/selection of the potential questions
~
You are absolutely right! It is virtually impossible to produce -all-
questions that could possibly try to understand/introspect/own some issue
~
thanx
lbrtchx |
|
| |
|
| Back to top |
Milind Joshi
Guest
|
| Posted: Tue Jun 24, 2008 4:54 pm Post subject: Re: Mining Questions and Answers in Texts |
|
|
On Jun 20, 7:37 pm, Albretch Mueller <lbrt...@gmail.com> wrote:
| Quote: | Cameron Hughes wrote:
If you're not asking about 'identifying questions in the text' then
could clarify what you
mean by 'mining questions in the text'
I'm sure you don't mean that you're trying to extrapolate what questions
can be asked
from some given text, because that could be infinitely many questions.
So please clarify...
~
Well, you helped me see my bias
~
The thing is that right now I am trying to extract questions and answers
from technical specifications and of course, since they are such
(specifications) you may just get all conjugations of the verb to be and do
the parsing/selection of the potential questions
~
You are absolutely right! It is virtually impossible to produce -all-
questions that could possibly try to understand/introspect/own some issue
~
thanx
lbrtchx
|
One technique, that has had some limited success is to build parse
trees and identify each word with it's POS tag. The same parse trees
can then be turned upside down, but in a toy project I played with
some time ago, it produces some crazy questions, and the answers could
often be out-of-context because there's a lot more to a technical
specification than plain grammar or Part-of-Speech tags... the
semantics come into play here. Sometimes, a human expert will copy-
paste text from different parts of the documentation to answer a given
question, and sometimes, they just improvise an answer from the
information that is in the documentation.
You can start playing with POS taggers, and there's a tremendous
amount of work done on parsers and grammars in the Computational
Linguistics Research world.
Also note that some approaches are more language-dependent than
others.
Another technique that comes to mind is not to bother trying to search
for questions before-hand, for reasons you and Cameron Hughes have
discussed before. Instead, when a user types a question, use the terms
in the question to generate answers from known texts... yahoo answers,
and ask.com and other question-answering services work in this manner.
They also use a combination auto/semi-automatic/manual approaches to
finding the best answers to questions - some sites simply bring askers
and answerers together.
A good basic textbook in this area to get you up-and-running (if
you're not already familiar) is "Foundations of statistical natural
language processing" by Manning & Schütze, MIT Press.
Hope this helps!
Best Regards,
Milind Joshi |
|
| |
|
| Back to top |
Cameron Hughes
Guest
|
| Posted: Tue Jul 01, 2008 5:17 pm Post subject: Re: Mining Questions and Answers in Texts |
|
|
Albretch Mueller wrote:
| Quote: | Cameron Hughes wrote:
If you're not asking about 'identifying questions in the text' then
could clarify what you
mean by 'mining questions in the text'
I'm sure you don't mean that you're trying to extrapolate what questions
can be asked
from some given text, because that could be infinitely many questions.
So please clarify...
~
Well, you helped me see my bias
~
The thing is that right now I am trying to extract questions and answers
from technical specifications and of course, since they are such
(specifications) you may just get all conjugations of the verb to be and do
the parsing/selection of the potential questions
~
You are absolutely right! It is virtually impossible to produce -all-
questions that could possibly try to understand/introspect/own some issue
~
thanx
lbrtchx
|
A legitmate approach to building a Q&A system can be found in
"Natural Language Question Answering System" by Boris Galitsky
This book gives some very doable architectures for real world systems.
Subject matter such as "technical specifications" are very good candidates
for his "Semantic Headers" technique.
Cheers
Cameron |
|
| |
|
| Back to top |
pgfulg
Guest
|
| Posted: Sat Nov 22, 2008 11:11 am Post subject: BHAgQUxxjzNlyC |
|
|
XCu8xc <a href="http://npmtgpmhcdlh.com/">npmtgpmhcdlh</a>,
Linking to external sites is not allowed,
[link=http://rkyeutafmrlu.com/]rkyeutafmrlu[/link],
http://tewhmnkcgabi.com/ |
|
| |
|
| Back to top |
|
