`

java 语义分析 开源

java自然语言分析 
阅读更多

 

语义分析 

LingPipe的优势是:

  • 比较全面的覆盖自然语言处理的各个分支，文本分词，聚类，语义情感分析，领域知识学习等等
  • 具有全套在research上免费的源码，样列代码，测试代码(商业与非商业均同一套代码)，并且文档详细，对于其中模型所参考的论文都引用出来，适合研究学习.
  • 作为相对开源资源缺少的领域，项目一直持续更新中.

包含的模块：

  • 主题分类（Top Classification） : 基于文本语言模型训练，归类
  • 命名实体识别（Named Entity Recognition）:基于first-best, n-best and per-entity confidencemodes识别，以及训练与评估识别器
  • 聚类（Clustering）: 基于single-link andcomplete-link多层聚类，包裹一些聚类评估技术
  • 词性标注（Part-of Speech Tagging）:
  • 句题检测（Sentence Detection）:
  • 拼写更正（Spelling Correction）:基于"你要找的是"风格的检查引擎
  • 数据库文本挖掘（Database Text Mining）
  • 字符串比较(String Comparison) ：基于距离与相似度测量，包括权重距离，TF/IDF距离，Jaccard distance, Jaro-Winkler distance,等
  • 兴趣短语检测（Interseting Phrase Detection）
  • 字符语言建模（Character Language Modeling）
  • 中文分词（Chinese WordSegmentation）基于空格分割类似训练库，机器学习，发现认知新词
  • 数据库文本挖掘（Database Text Mining）
  • 情感分析（Sentiment Analysis）基于文本聚类
  • 断字识音（Hyphenation and Syllabification）
  • 语言辨别（Language Identification）
  • 奇异值分解（Singular Value Decomposition）
  • 逻辑回归 （Logistic Regression）
  • 期望最大化（Expectation Maximization）
  • 词义排歧（Word Sense Disambiguation）

LingPipe包含资源:

 

  • Papaer&language material :source，介绍中均包含有所引用资源

目前个人应用LingPipe包中的中文分词，结合情感分析模块研究中文情感检测与辨别。API接口均已高度概括化，便于快速实现，不过所运用的算法需要详尽的分析。

 

 

 

 


  • 自然语言工具包 NLTK

     

    NLTK 会被自然地看作是具有栈结构的一系列层，这些层构建于彼此基础之上。那些熟悉…

     

 

 

 


  • 机器学习软件包 Mallet

     

    Mallet是专门用于机器学习方面的软件包，此软件包基于java。通过mallet工具，可以进…

     

 

 


  • DKPro Core

     

    DKPro Core 是基于 Apache UIMA 框架之上的自然语言处理（NLP）的软件组件。DKPro…

     

 


  • TextTeaser

     

    TextTeaser是一个自动摘要算法,结合了自然语言处理的力量和机器学习产生好结果。…

 

OpenNLP

OpenNLP是一个基于Java机器学习工具包，用于处理自然语言文本。支持大多数常用的 NLP 任务，例如：标识化、句子切分、部分词性标注、名称抽取、组块、解析等。

FudanNLP

FudanNLP主要是为中文自然语言处理而开发的工具包，也包含为实现这些任务的机器学习算法和数据集。本工具包及其包含数据集使用LGPL3.0许可证。开发语言为Java。
功能：
1. 文本分类 新闻聚类
2. 中文分词 词性标注 实体名识别 关键词抽取 依存句法分析 时间短语识别
3. 结构化学习 在线学习 层次分类 聚类 精确推理

Standford NLP

Standford NLP提供了一系列的自然语言处理工具。

 

机器学习

Support Vector Machine

An implementation of Vapnik's Support Vector Machine A Library for Support Vector Machines

Decision Tree

The "classic" decision-tree tool, developed by J. R. Quinlan Tutorial

Maximum Entropy

Yet Another Small MaxEnt Toolkit

Conditional Random Field

A simple, customizable, and open source implementation of Conditional Random Fields (CRFs) for segmenting/labeling sequential data

自然语言处理

综合

An organizational center for open source projects related to natural language processing A suite of UNIX software tools to facilitate the construction and testing of statistical language models A Java-based development package for academic use in information retrieval (IR) and text mining. Include many NLP tools A suite of Java libraries for the linguistic analysis of human language, including
  • track mentions of entities (e.g. people or proteins);
  • link entity mentions to database entries;
  • uncover relations between entities and actions;
  • classify text passages by language, character encoding, genre, topic, or sentiment;
  • correct spelling with respect to a text collection;
  • cluster documents by implicit topic and discover significant trends over time; and
  • provide part-of-speech tagging and phrase chunking.
Open source Python modules, linguistic data and documentation for research and development in natural language processing and text analytics, with distributions for Windows, Mac OSX and Linux.
  • Advanced Natural Lange Object-oriented Processing Environment.包括一系列工具(特别c#的stanford parser)

分词

中科院的中文分词系统 A Java implementation of a CRF-based Chinese Word Segmenter

词性标注

A error-driven transformation-based tagger implemented by Eric Brill A Java implementation of the log-linear part-of-speech taggers descriped by Kristina Toutanova, et.al. A decision tree based tagger from the University of Stuttgart. An HMM-based Java POS tagger from Birmingham U.

命名实体识别

A Java implementation of a Conditional Random Field sequence model, together with well-engineered features for Named Entity Recognition Tools include statistical named-entity recognition, a heuristic sentence boundary detector, and a heuristic within-document coreference resolution engine. Java. GPL. By Bob Carpenter, Breck Baldwin and co. SVM-based NP-chunker, also usable for POS tagging, NER, etc. C/C++ open source. Won CoNLL 2000 shared task. (Less automatic than a specialized POS tagger for an end user.)

Stemming

A process for removing the commoner morphological and inflexional endings from words in English byMartin Porter A small string processing language designed for creating stemming algorithms for use in Information Retrieval.

句法分析

Java implementations of probabilistic natural language parsers, both highly optimized PCFG and dependency parsers, and a lexicalized PCFG parser.

文本挖掘

摘要

其他

加密

包括众多加密算法，RSA、DES、MD5、SHA等 Win32安装版

压缩

A Massively Spiffy Yet Delicately Unobtrusive Compression Library

日志

Creates and maintains open-source software related to the logging of application behavior and released at no charge to the public, including 注: log4cxx官方版本有内存泄漏问题

Unicode

A mature, widely used set of C/C++ and Java libraries providing Unicode and Globalization support for software applications

XML

A validating XML parser, including C and Java edition

多字符串匹配

  • AC in C# : Aho-Corasick string matching in C#

HTML Parser

  • Html Agility Pack , an agile HTML parser that builds a read/write DOM and supports plain XPATH or XSLT. It is a .NET code library that allows you to parse "out of the web" HTML files.
  • Majestic-12 , an open source high-performance .NET C# module that was created to parse HTML for links, indexing and other purposes. 速度快，但不生成dom树

外部联接

 

自然语言处理

http://www.oschina.net/p/lingpipe/similar_projects

 

http://opennlp.apache.org/cgi-bin/download.cgi

 

https://code.google.com/p/fudannlp/

分享到：
| Lucene使用IKAnalyzer分词实例 及 IKAnaly ...
评论
发表评论

您还没有登录,请您登录后再发表评论

相关推荐

Magicbox
Global site tag (gtag.js) - Google Analytics