Posts

Showing posts with the label R

conceptual question about FDR, FDR adjusted p-value and q-value

Dear Jack, The thing to understand is that terms like FDR and q-value were defined in specific ways by their original inventors but are used in more generic ways by later researchers who adapt, modify or use the ideas. The term "false discovery rate (FDR)" was created by Benjamini and Hochberg in their 1995 paper. They gave a particular definition of what they meant by FDR. Their procedure accepted or rejected hypotheses, but did not produce adjusted p-values. Benjamini and Yekutieli presented another more conservative algorithm to control the FDR in a 2001 paper. Same definition of FDR, but a different algorithm. In 2002, I re-interpreted the Benjamini and Hochberg (BH) and Benjamini and Yekutieli (BY) procedures in terms of adjusted p-values. I implemented the resulting algorithms in the function p.adjust() in the stats package, and used them in the limma package, and this lead to the concept of an FDR adjusted p-value. The terminology used by the p.adj...

NotFromMe: stringsAsFactors: An unauthorized biography

This is not written by me. I just copy and paste it here. The origninal link is : http://simplystatistics.org/2015/07/24/stringsasfactors-an-unauthorized-biography/ Recently, I was listening in on the conversation of some colleagues who were discussing a bug in their R code. The bug was ultimately traced back to the well-known phenomenon that functions like ‘read.table()’ and ‘read.csv()’ in R convert columns that are detected to be character/strings to be factor variables. This lead to the spontaneous outcry from one colleague of Why does stringsAsFactors not default to FALSE???? The argument ‘stringsAsFactors’ is an argument to the ‘data.frame()’ function in R. It is a logical that indicates whether strings in a data frame should be treated as factor variables or as just plain strings. The argument also appears in ‘read.table()’ and related functions because of the role these functions play in reading in table data and converting them to data frames. By default, ‘strin...

R tip: read.table or read.csv a table with quotes

Error message: Warning message: In scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,  :   EOF within quoted string Answer: You need to disable quoting. cit <- read.csv ( "citations.CSV" , quote = "" , row.names = NULL , stringsAsFactors = FALSE ) str ( cit ) ## 'data.frame': 112543 obs. of 13 variables: ## $ row.names : chr "10.2307/675394" "10.2307/30007362" "10.2307/4254931" "10.2307/20537934" ... ## $ id : chr "10.2307/675394\t" "10.2307/30007362\t" "10.2307/4254931\t" "10.2307/20537934\t" ... ## $ doi : chr "Archaeological Inference and Inductive Confirmation\t" "Sound and Sense in Cath Almaine\t" "Oak Galls Preserved by the Eruption of Mount Vesuvius in A.D. 79_ and Their Probable Use\t" "The Arts Four Thousand Years Ago\t" ....

R: DESeq2 analysis: outliers and refitting

When I was running DESeq2, I got the message shown as below: converting counts to integer mode estimating size factors estimating dispersions gene-wise dispersion estimates mean-dispersion relationship final dispersion estimates fitting model and testing -- replacing outliers and refitting for 47 genes -- DESeq argument 'minReplicatesForReplace' = 7 -- original counts are preserved in counts(dds) estimating dispersions fitting model and testing I didn't encounter this before. Here are the reasons: Answers: The count outlier flagging is useful when there are a minority of outliers in the dataset, but as you have noted, something else is going on here with so many genes flagged. There are two reasons for so many genes being flagged as outlier: either the method for flagging outliers is not appropriate for the distribution of counts in your data and should be turned off (by setting minReplicatesForReplace=Inf and cooksCutoff=FALSE), or you have a sample...

R_chart.Correlation

Image
The chart.Correlation() function from the PerformanceAnalytics package produces a very nice scatterplot matrix, with histograms, kernel density overlays, absolute correlations, and significance asterisks (0.05, 0.01, 0.001). mydata <- mtcars [ , c ( 1 , 3 , 4 , 5 , 6 ) ] # plot the data library ( PerformanceAnalytics ) chart.Correlation ( mydata )