JWPL


JWPL Bug:
Bug:org.xml.sax.saxparseexception JAXP00010004

JWPL Debug:
Copy this code in App.java


package maven.maven;

import javax.xml.XMLConstants;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.parsers.SAXParserFactory;

import org.xml.sax.SAXNotRecognizedException;
import org.xml.sax.SAXNotSupportedException;

import de.tudarmstadt.ukp.wikipedia.datamachine.domain.JWPLDataMachine;

public class App
{
    public static void main( String[] args ) throws SAXNotRecognizedException, SAXNotSupportedException, ParserConfigurationException
    {
         System.setProperty("jdk.xml.totalEntitySizeLimit", "500000000");
        SAXParserFactory spf = SAXParserFactory.newInstance();
       spf.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, false);
        // Path where wiki dump is stored.
        String[] arg = {"french", "Accueil", "Homonymie", "/home/qcrialt/ATQAM/WikipediaDumps/frwiki-20170920"};
        JWPLDataMachine.main(arg);
        // french Accueil Homonymie
       // /home/qcrialt/ATQAM/WikipediaDumps/frwiki-20170920

    }
}

Add this dependency in pom.xml
https://dkpro.github.io/dkpro-jwpl/DeveloperSetup/ 



<dependency>
  <groupId>de.tudarmstadt.ukp.wikipedia</groupId>
  <artifactId>de.tudarmstadt.ukp.wikipedia.datamachine</artifactId>
  <version>1.0.0</version>
</dependency>
 
  
 link to JWPL tables.sql
 
Seconn 
 
JWPL Working time for English= 83.826.173 ms
JWPL Working time for French = 8.245.844 ms
JWPL Working time for Arabic= 2.004.813 ms 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Comments

Post a Comment

Popular posts from this blog

Links

DISEQuA Multisix corpus Bitbucket Account JWPL DataMachine Book:Speech and Language Processing(IR, IE, linguistic definitions) An example of a bilingual corpus Datasets for Natural Language Processing RNN QCRI Live Translator QCRI MT QCRI MT Article Multilingual QAS (steps explanation) Link for QA English datasets TF-IDF Links From Dr. Alberto Book : Machine Translation evaluation-Likert Scale Corpora referneces - QCRI -Memex "DISEQuA" and "Multisix" corpora
Read more

Wikipedia Corpora

Image
 To Do : Create a table comparing WikiQA and SQuAD and say we are going to choose WikiQA (for the moment) WikiQA:  WikiQA is a set of question and sentence pairs, collected and annotated for research on open-domain QA. It includes questions for which there are no correct sentences, enabling researchers to work on answer triggering. WikiQA introduced the task of answer triggering and was the only answer triggering dataset. Questions type The questions are originally sampled from  Bing query logs . This corpus has 3047 questions (in raw data file   WikiQA.tsv  ). It contains both general and factoid questions.  -HOW: eg.  how did Athenians make money? how does interlibrary loan work? ( NO WHY Questions ) -What, How many, Who, when, Where etc. Date/Version  Version 1.0: August 25, 2015 Answer triggering - We propose the answer triggering task, a new challenge for the question answering problem, which requires QA s...
Read more

Definitions & How to

extract summary section from Wikipedia (Python) Generating a Plain Text Corpus from Wikipedia What is TSV? What Opens a TSV? How to import TSV file in MS Excel What is a pickle file? “Pickling” is the process whereby a Python object hierarchy is converted into a byte stream, and “unpickling” is the inverse operation, whereby a byte stream (from a binary  file  or bytes-like object) is converted back into an object hierarchy. Generating a Plain Text Corpus from Wikipedia remove HTML from Wikipedia text JAVA LIBS FOR PROCESSING WIKI MARKUP Remove Markup from wiki text Mylyn Wikitext example (plain java, using maven) de.tudarmstadt.ukp.wikipedia.parser A Java Wikipedia markup to plain text converter eclipse link Python library to extract summary section of Wikipedia article SQUAD Reformatter -Python Machine translation evaluation book
Read more