textmineR 3.0.4 Unreleased

This version is a patch. In this version I have

• Removed unconditional stripping in MAKEVARs as specified by CRAN
• Improved outputs of FitLdaModel

textmineR 3.0.3 2019-03-22

This version is a patch. In this version I have

• fixed an error related to the update.lda_topic_model method.
• added a method posterior.lda_topic_model to sample from the posterior of an LDA topic model.

textmineR 3.0.2 2019-01-09

This version is a patch. In this version I have

• changed some elements of NAMESPACE to pass additional CRAN checks.
• added an update method for the lda_topic_model class. This allows users to add documents to an existing model (and even add new topics) without chaning the indices of previously-trained topics. e.g. topic 5 is still topic 5.
• added a vignette for using tidytext alongside textmineR

textmineR 3.0.1 2018-10-31

This version is a patch in response to issues revealed by automatic checks upon submission to CRAN plus an additional issue I encountered along the way.

I have

• Used the CRAN template for my MIT LICENSE file
• Modified the example of the LabelTopics function to speed up run time for that example
• Modified vignettes to run in less time
• Added a Makevars file to keep compiled code small on Ubuntu.

textmineR 3.0.0 Unreleased

This version significantly changes textmineR.

• Several functions that were slated for deletion in version 2.1.3 are now gone.
• RecursiveRbind
• Vec2Dtm
• JSD
• HellDist
• GetPhiPrime
• FormatRawLdaOutput
• Files2Vec
• DepluralizeDtm
• CorrectS
• CalcPhiPrime
• FitLdaModel has changed significantly.
• Now only Gibbs sampling is a supported training method. The Gibbs sampler is no longer wrapping lda::lda_collapsed_gibbs_sampler. It is now native to textmineR. It’s a little slower, but has additional features.
• Asymmetric priors are supported for both alpha and beta.
• There is an option, optimize_alpha, which updates alpha every 10 iterations based on the value of theta at the current iteration.
• The log likelihood of the data given estimates of phi and theta is optionally calculated every 10 iterations.
• Probabilistic coherence is optionally calculated at the time of model fit.
• R-squared is optionally calculated at the time of model fit.
• Supported topic models (LDA, LSA, CTM) are now object-oriented, creating their own S3 classes. These classes have their own predict methods, meaning you do not have to do your own math to make predictions for new documents.

• A new function SummarizeTopics has been added.

• tm is no longer a dependency for stopwords. We now use the stopwords package. The extended result of this is that there is no longer any Java dependency.

• Several packages have been moved from “Imports” to “Suggests”. The result is a faster install and lower likelihood of install failure based on packages with system dependencies. (Looking at you, topicmodels!)

• Finally, I have changed the textmineR license to the MIT license. Note, however, that some dependencies may have more restrictive licenses. So if you’re looking to use textmineR in a commercial project, you may want to dig deeper into what is/isn’t permissable.

textmineR 2.1.3 2018-09-11

• Deprecating functions that will be removed, renamed, or have significant changes to syntax or functionality in the forthcoming textmineR v3.0.
• Functions slated for deletion:
• RecursiveRbind
• Vec2Dtm
• JSD
• HellDist
• GetPhiPrime
• FormatRawLdaOutput
• Files2Vec
• DepluralizeDtm
• CorrectS
• CalcPhiPrime
• In addition: FitLdaModel is going to change significantly in its functionality and argument calls.

textmineR 2.1.2 2018-04-29

• Deprecated RecursiveRbind - it depended on a deprecated function from the Matrix package. And the replacement offered by Matrix operates recursively, making this function truly superfluous.

textmineR 2.1.1 2018-03-06

• Corrected some code in the vignettes that caused errors on Linux machines.

textmineR 2.1.0 Unreleased

• Added vignettes for common use cases of textmineR
• Modified averaging for CalcProbCoherence
• Updated documentation to CreateTcm

textmineR 2.0.6 2017-08-17

• Back-end changes to CreateTcm in response to new text2vec API. Functionality is unchanged.
• Changes to how the package interfaces with Rcpp

textmineR 2.0.5 2017-04-07

• Add verbose option to CreateDtm and CreateTcm to surpress status messages.
• Add function GetVocabFromDtm to get text2vec vocabulary object from a dgCMatrix document term matrix.

textmineR 2.0.4 2016-11-03

• Patching errors introduced in version 2.0.3

textmineR 2.0.3 2016-10-06

• Patches to CreateDtm and CreateTcm in response to updates to text2vec.
• More formal update to take advantage of text2vec’s latest optimizations to follow.

textmineR 2.0.2 2016-06-06

• Patched CreateDtm and CreateTcm. remove_punctuation now supports non-English characters.
• Patched TmParallelApply. Added an option to declare the environment to search for your export list. Default to that argument just searches the local environment. The default should cover ~95% of use cases. (And avoids crash on Windows OS)
• Patched FitLdaModel. Use of the ... argument now allows you to control TmParallelApply, lda::lda.collapsed.gibbs.sampler, and topicmodels::LDA without error.
• Patched FitCtmModel where the ... argument now goes to topicmodels::CTM’s control argument.
• Patched CreateTcm to return objects of class dgCMatrix. This allows you to run functions like FitLdaModel on a TCM.
• Switched from irlba to RSpectra for LSA models because RSpectra’s implementation is much faster.

textmineR 2.0.1 2016-04-24

• Patched CreateDtm and CreateTcm. An error caused stopwords to not be removed

textmineR 2.0.0 2016-04-19

• Vec2Dtm is now deprecated in favor of CreateDtm
• A function, CreateTcm, now exists to create term co-occurence matrices
• CreateDtm and CreateTcm are implemented with a parallel C++ back end through the text2vec library
• the implementation is much faster! I’ve clocked 2X - 10X speedups, depending on options
• adds external dependencies - C++ compiler and GNU make - and takes away an external dependency - Java.
• now all tokens will be included, regardless of length. (tm’s framework silently dropped all tokens of fewer than 3 characters.)
• Allow generic stemming and stopwords in CreateDtm & CreateTcm
• Now there is only one argument for stopwords, making it clearer how to use custom or non-English stopwords
• Now the stemming argument allows for passing of stem/lemmatization functions.
• Function for fitting correlated topic models
• Function to turn a document term matrix to term co-occurence matrix
• Allowed LabelTopics to use unigrams, if you want. (n-grams are still better.)
• More robust error checking for CalcTopicModelR2 and CalcLikelihood
• All function arguments use "_“, not”.".
• CalcPhiPrime replaces (the now deprecated) GetPhiPrime
• Allows you to pass an argument to specify non-uniform probabilities of each document
• Similarly, CalcHellingerDist and CalcJSDivergence replace HellDist and JSD. This is to conform to a naming convention where functions are “verbs”.

textmineR 1.7.0 2016-03-31

• Added modeling capability for latent semantic analysis in FitLsaModel()
• Added CalcProbCoherence() function which replaces ProbCoherence() and can calculate probabilistic coherence for the whole phi matrix.
• Added data from NIH research grants instead of borrowd data from tm
• Removed qcq data
• Added variational em method for FitLdaModel()
• Added function to represent document clustering as a topic model Cluster2TopicModel()

textmineR 1.6.0 2016-03-06

• Add deprecation warning to ProbCoherence
• Allow for arguments of number of cores to be passed to every function that uses implicit parallelziation
• Allow for passing of libraries to TmParallelApply (makes this function truely independent of textmineR)
• For Vec2Dtm ensure that stopwords and custom stopwords are lowercased when lower = TRUE
• Update README example to use model caches