JWPL


JWPL Bug:
Bug:org.xml.sax.saxparseexception JAXP00010004

JWPL Debug:
Copy this code in App.java


package maven.maven;

import javax.xml.XMLConstants;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.parsers.SAXParserFactory;

import org.xml.sax.SAXNotRecognizedException;
import org.xml.sax.SAXNotSupportedException;

import de.tudarmstadt.ukp.wikipedia.datamachine.domain.JWPLDataMachine;

public class App
{
    public static void main( String[] args ) throws SAXNotRecognizedException, SAXNotSupportedException, ParserConfigurationException
    {
         System.setProperty("jdk.xml.totalEntitySizeLimit", "500000000");
        SAXParserFactory spf = SAXParserFactory.newInstance();
       spf.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, false);
        // Path where wiki dump is stored.
        String[] arg = {"french", "Accueil", "Homonymie", "/home/qcrialt/ATQAM/WikipediaDumps/frwiki-20170920"};
        JWPLDataMachine.main(arg);
        // french Accueil Homonymie
       // /home/qcrialt/ATQAM/WikipediaDumps/frwiki-20170920

    }
}

Add this dependency in pom.xml
https://dkpro.github.io/dkpro-jwpl/DeveloperSetup/ 



<dependency>
  <groupId>de.tudarmstadt.ukp.wikipedia</groupId>
  <artifactId>de.tudarmstadt.ukp.wikipedia.datamachine</artifactId>
  <version>1.0.0</version>
</dependency>
 
  
 link to JWPL tables.sql
 
Seconn 
 
JWPL Working time for English= 83.826.173 ms
JWPL Working time for French = 8.245.844 ms
JWPL Working time for Arabic= 2.004.813 ms 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Comments

Post a Comment

Popular posts from this blog

Wikipedia Corpora

Image
 To Do : Create a table comparing WikiQA and SQuAD and say we are going to choose WikiQA (for the moment) WikiQA:  WikiQA is a set of question and sentence pairs, collected and annotated for research on open-domain QA. It includes questions for which there are no correct sentences, enabling researchers to work on answer triggering. WikiQA introduced the task of answer triggering and was the only answer triggering dataset. Questions type The questions are originally sampled from  Bing query logs . This corpus has 3047 questions (in raw data file   WikiQA.tsv  ). It contains both general and factoid questions.  -HOW: eg.  how did Athenians make money? how does interlibrary loan work? ( NO WHY Questions ) -What, How many, Who, when, Where etc. Date/Version  Version 1.0: August 25, 2015 Answer triggering - We propose the answer triggering task, a new challenge for the question answering problem, which requires QA s...
Read more

Links

DISEQuA Multisix corpus Bitbucket Account JWPL DataMachine Book:Speech and Language Processing(IR, IE, linguistic definitions) An example of a bilingual corpus Datasets for Natural Language Processing RNN QCRI Live Translator QCRI MT QCRI MT Article Multilingual QAS (steps explanation) Link for QA English datasets TF-IDF Links From Dr. Alberto Book : Machine Translation evaluation-Likert Scale Corpora referneces - QCRI -Memex "DISEQuA" and "Multisix" corpora
Read more

Extraction Model

Extraction Model:A nswer sentence selection Due to increasing complexity in question answering, deep learning has become a popular trend in solving difficult problems. Approaches Word Count:  counts the number of non-stopwords in the question that also occur in the answer sentence.  Weighted Word Count:  re-weights the counts by the IDF values of the question words. LCLR :makes use of rich lexical semantic features, including word/lemma matching, WordNet and vector-space lexical semantic models. Convolutional Neural Network Recurrent Neural Network
Read more