312 questions
0
votes
1
answer
90
views
Correlate2d using pearson zscore normalization
Given two 2d arrays I want to calculate the cross-correlation similar as done in scipy.signal.correlate2d but instead of using a fill_value=0 I want to calculate the Pearson R from the data overlap of ...
- 170
0
votes
1
answer
90
views
R correlation: I'm getting inconsistent correlation results with cor() function
I'm getting inconsistent results when using cor() function. It is easiest to demonstrate with small piece of code:
data("pbc2.id", package = "JM") # Mayo Clinic Primary Biliary ...
- 359
1
vote
2
answers
97
views
How to generate a sample with a given spearman coefficient
In order to create a dataset to test a statistical calculation package, I want to be able to generate a sample that is correlated to a reference sample with a given searman coefficient.
I managed to ...
- 647
0
votes
1
answer
47
views
How can I change the confidence interval of a forest plot? the at-function isn't working
I have tried everything: Different functions and changed the way I type in the confidence interval (CI) that I want. I need a CI for correlations that ranges from -1 to 1 by 0.5 steps.
my R-code:
data ...
- 3
0
votes
0
answers
32
views
Correlational analysis focusing on one variable and indicating significant values
I am able to make a correlation matrix that is formatted with the significant values and stars indicating how significant but dont know how to make it so it focuses only on VOC.
Output I have:
...
0
votes
0
answers
71
views
How to add superscript and subscript in correlation matrix using metan library in r studio
I have used the following code in correlation matrix using metan library in r studio,
library(metan)
View(CorrawpomF)
All <- corr_coef(CorrawpomF)
all
plot(All)
and the output of the code is like ...
- 1
0
votes
2
answers
81
views
Determine which variables in dataframe are most correlated to two other variables
Say that I have a sample dataframe like so.
sample_df <- data.frame(replicate(100,sample(0:1,1000,rep=TRUE)))
One thing you could ask is which columns in the dataframe are most correlated to ...
- 195
0
votes
1
answer
287
views
How to use Pearson's correlation in boostrap in python?
I have task to do in python. Here is the description:
Using the function scipy.stats.bootstrap(), perform interval estimation of the following parameters at a significance level of 1-alpha = 0.98 ...
- 11
0
votes
0
answers
107
views
Getting "ValueError: array must not contain infs or NaNs" even after checking that NaN and infs are 0
I've a data frame of 2942 rows and 52 columns. One of the columns is the target variable and the other columns are features. I'm trying to calculate Pearson's correlation between the feature column(s) ...
1
vote
0
answers
58
views
What is incorrect in my autocorrelation computation?
In the following source codes, Python and R are giving the same result.
However, C# is giving an incorrect result.
How can I fix the C# code?
Input data:
[1.0, 1.25, 1.5, 1.75, 2.0, 2.25, 3.33]
...
- 17.5k
0
votes
1
answer
125
views
How to find if two arrays are from uncorrelated distributions?
I have two arrays - array1 and array2. How confidently can I state that they are generated from uncorrelated distributions? Just checking the correlation values would not suffice; I would require p-...
0
votes
1
answer
146
views
Force intercept at origin with scipy.stats.pearsonr or alternatives?
I checked the documentation of scipy.stats.pearsonr but did not find any options to force intercept at origin 0,0.
If it's impossible with scipy.stats.pearsonr does anyone know any alternatives?
- 495
0
votes
2
answers
549
views
How to increase the font size of auto-generated R and p-value in ggscatter plot?
I want to increase the font size of autogenerated r2 and p value. I used the code like
+ theme(text = element_text(size = 18)) but it did not make any changes?
The code used are given below
# Load ...
- 589
0
votes
2
answers
71
views
How to use for loop to perform Pearson correlation in r
I have a dataset 673 x 232. I want to use for loop to perform Pearson correlation and save p-value and estimate.
My code is
vec_lipid <- colnames(df3[,9:232])
df4 <- data.frame(vec_lipid)
df4$p....
- 21
0
votes
1
answer
237
views
Is Pearson correlation faster than Spearman correlation in R?
I would like to determine many correlations (millions) between pairs of columns, so I am worried about computing time.
I suspect that Pearson correlations (based on values) are faster to calculate in ...
- 1,383
0
votes
2
answers
79
views
.corr() method for dataframe not returning ideal values only returns either -1 or 1
Ideally it should be returning values between -1 and 1 for every cell except for the cells that have the same column name and row name those need to have a 1 value
Tried replacing the NaN with 0 ...
0
votes
0
answers
145
views
remove y=0 values and x=0 values in scatter plot in R
I have to do linearity test between different pair of variables. I'm using the code ggpairs(data) to do it as I have multiple variables. But many of my variables have y=0 values and x=0 values. So my ...
- 25
1
vote
1
answer
104
views
x-axis text disappears after customizing ggpairs plot
I want to create a correlation matrix with ggpairs. I only need the lower part. The upper and diagonal parts are not relevant. Furthermore, I want to keep the final plot simple and therefore remove ...
- 13
0
votes
1
answer
143
views
Add annotation bars instead of text to axis of a correlation heatmap created with ggplot
Hi Thank you for reading and potential help!
I am trying to annotate the x and y-axis of a corelation heatmap generated using ggplot with the following code:
# Gen ggplot
ggplot(cor.df, aes(x, y, fill ...
0
votes
1
answer
63
views
Why are there letters in my correlation table in Pandas? [duplicate]
I've printed a correlation table on Pandas for some results derived from a questionnaire. There are some letters in the correlation values for some reason. Not sure what's causing this and how to ...
- 11
1
vote
0
answers
57
views
LDA & CVA biplots show arrows in opposite directions but correlation matrix shows strong positive correlation
I performed an LDA for 50 elements vis-a-vis land cover type on R. The final plot showed 13 elements which contributed the most to the separability of the 5 land cover types. However, when I created a ...
- 11
0
votes
1
answer
112
views
Eliminate redundant data from Pearson correlation to R
Having made a correlation of Pearson on R, I would like to simplify my data set by selecting the indicators with a correlation, between two, greater than 0.7 and less than -0.7 and by eliminating ...
- 1
0
votes
1
answer
73
views
How to perform a correlation test in a loop, while removing a row from the dataframe in every iteration?
I am trying to remove a row from my dataframe in every iteration in a for loop and perform correlation test on the newly saved dataframe. However, I am not getting what I expect. Please help. Each row ...
1
vote
1
answer
154
views
How is the correlation matrix using cor calculated?
I implemented my own correlation function in R. Surprisingly I get slightly different results when using the built-in cor function. The differences seem to disappear when n the number of observations ...
- 6,472
2
votes
1
answer
98
views
Scatter plot toy examples to benchmark a correlation coefficient
I am interested in benchmarking a coefficient and would like to see some toy examples. I came across this link which includes the following image. Would anyone happen to know of a Python toolkit or be ...
- 1,041
0
votes
1
answer
57
views
Need help complex correlation analysis (150 columns against 36)
I have to carry out a relatively complex correlation analysis, in which a total of 150 columns are to be examined of correlation against 36 other columns. I have already done this for 30 columns ...
- 37
0
votes
0
answers
211
views
Hierarchical clustering, problem with distance metric(Pearson, Manhattan, Euclidean)
I am facing this strange problem, that i am not able to figure out. With euclidean distance and manhattan distance(either their are standardized or not), clusters are divided in very strange way.
I ...
- 11
0
votes
1
answer
170
views
Performing a correlational analysis on ASV tables and Metadata tables
I am extremely nervous to post because I am a beginner and took on a bit more than I could chew. Nonetheless, the problem is that I am working on a project where I write a script to correlate the ...
0
votes
1
answer
37
views
Facetted area plot of cumulative proportion with trend lines and Pearson's correlation coefficient?
I want to create a facetted area plot showing the cumulative proportion of each category, with a trend line and a Pearson's correlation coefficient.
I've so far used this code (exemplified with the ...
- 429
0
votes
1
answer
988
views
Pearson correlation coefficient for complex time series
I would like to know how to calculate the Pearson correlation coefficient for two complex time series.
Do we simply do
Or there is something else?
import numpy as np
R = lambda x,y: ((x-x.mean())...
- 1,670
1
vote
1
answer
326
views
How to calculate a coefficient of correlation, by year, in R and put results in a dataframe?
I want to calculate the coefficient of correlation, by year, in R and put the results in a dataframe (then repeat the process by calculating the coefficient of determination). The following code ...
1
vote
0
answers
487
views
Principal Component Analysis and Correlation in R
Could you please explain how to use principal components in principal component analysis in a correlation analysis?
I performed a PCA on my dataset and extracted 2 components; Now I'm wondering how to ...
0
votes
0
answers
68
views
Heap based permutation extremely slow for larger numbers
We are using Heap algorithm for generating permutations of an array a. . The generated permutations are passed to the printArr function,
which calculates the correlation coefficient between two arrays ...
- 804
1
vote
1
answer
98
views
Correlation analysis between specific columns (in groups of three) possible?
It is possible to calculate the correlation coefficients and p-values in groups? For example with this dataset:
df<-read.csv("http://renatabrandt.github.io/EBC2015/data/varechem.csv", row....
- 37
0
votes
1
answer
895
views
How do I make a correlation matrix for each subset of a column of my pandas dataframe?
Here’s the head of my dataframe:
There are 100 different loggers and 10 different years. I want to subset the table by logger and find the Pearson correlation values for year by avg_max_temp, ...
- 427
1
vote
0
answers
85
views
Correlation coefficients at each grid point in python
I am trying to calculate correlation coefficients at each grid point in python. The dimension of two variables(variable1[t,y,x], variable2[t,y,x]) are same and I would like to get a correlation ...
- 11
1
vote
0
answers
163
views
K-means clustering with Pearson_Correlation
I'm trying to replicate a feature in Systat, which is k-means clustering with Pearson correlation. I saw that a package called ClusterR allows you to do this by specifying a bunch of parameters. I ...
0
votes
0
answers
46
views
Panel Dataset and Correlation - Stata
I have a panel dataset and I am having trouble using the correlation command to get the correlation between variables. Can anyone provide a code for Pearson goodness of fit correlation with a panel ...
0
votes
0
answers
176
views
Cross-lagged Pearson correlation in R
I have a dataset where I have two recordings (sessions) of two different variables.
set.seed(123)
data <- data.table(
id = rep(1:20, each = 2),
session = rep(1:2, times = 20),
var1 = ...
- 499
1
vote
1
answer
136
views
How to slice and calculate the pearson correlation coefficient between one big and small array with "overlapping" windows arrays
Suppose I have two very simple arrays with numpy:
import numpy as np
reference=np.array([0,1,2,3,0,0,0,7,8,9,10])
probe=np.zeros(3)
I would like to find which slice of array reference has the highest ...
- 2,799
0
votes
1
answer
385
views
Computing the Pearson Correlation coefficient between a set of model weights trained in Keras
I am training an ImageNet-pretrained VGG-16 model in Keras with two different hyperparameter settings. I prefer to see if there exists a linear relationship between the two sets of model weights. For ...
- 1,189
1
vote
1
answer
410
views
Compute correlations of several vectors
I have several pairs of vectors (arranged as two matrices) and I want to compute the vector of their pairwise correlation coefficients (or, better yet, angles between them - but since correlation ...
- 60.5k
0
votes
0
answers
57
views
Pearson Correlation Result
I have been carrying out some Pearson correlations. In some of the outcome tables I get the following results inside instead of a correlation value:
"NA", "NaN", "-Inf***"...
- 15
0
votes
1
answer
2k
views
Multiple Pearson Correlation with p-Values
I want to calculate a Pearson correlation between several columns. The solution JasonAizkalns posted in this thread is very useful for me.
df %>%
select_if(is.numeric) %>%
group_by(year) %&...
- 35
1
vote
1
answer
925
views
Increasing font size of auto-generated R and p-value
This's a rather straightforward question where I'd like to increase the font size of the automatically generated R and p-value for my correlation plot via ggscatter. I've tried using cex but doesn't ...
- 21
1
vote
1
answer
2k
views
How to create Correlation Heat Map of All Measure in Tableau?
I have Query with 10 Measures I am able to draw correlation heat map in Python using below?
import pandas as pd
import seaborn as sn
import matplotlib as mt
df = pd.read_sql('select statement')
sn....
- 1,970
-3
votes
1
answer
145
views
Why do i keep gettig an error message o this code
this is an IBM skill lab code. try running it and keeps getting a error
pearson_coef, p_value = stats.pearsonr(df['city-mpg'], df['price'])
print("The Pearson Correlation Coefficient is", ...
0
votes
0
answers
226
views
Is there a python code for point biserial correlation adjusting covariates?
I have a binary variable (which is either 0 or 1) and continuous variables. I have continuous variables that I should adjust as covariates. I would like to see the result of the point biserial ...
1
vote
1
answer
258
views
np.corrcoef returns only nan
i have an array
norm_array=np.array([[1, 133, 1, 5.73, 5.09, 11.12, 10.16, 3.38, 15, 3, 8, 7, 4, 5, 1,
6, 1, 2, 18, 12, 48],
[1, 185, 0, 4.34, 3.66, 18.23, 14.91, 0, 21, 15, 11, 2, 4, 4, 5,
...
- 25
1
vote
0
answers
208
views
Reduce time complexity of nested loop
I want to reduce the runtime of my algorithm. Can anyone suggest ways to reduce its complexity other than using threads or parallel computing.
Algorithm:
This algorithm aims to solve Pearson's r where ...
- 197