Need Stats Help

[LongLiveYorke 38]

So z just means how many sigmas a point is away from the mean, I guess. It seems like if you have a particular data set, you can simply calculate it. The interpretation of what you calculate may not be significant if we're not dealing with a normal distribution, but certainly one can write down the answer if need be. I would simply listen to Actuary on this one, he seems to know what he's talking about.

[Actuary 3]

I've just never seen a z score applied to a non-normal distribution. I don't see what the purpose is.

To see how many Stnd Dev you are from the mean

It's standard practice that when you don't know the shape of the distribution you use a test that doesn't make assumptions about the underlying distribution of the data.

What if you know the shape?

[vbnautilus 48]

Edit: The dataset has something like an N of 16000...not sure if there's any difference between t-score and z-score for non-normal distributions, but I thought I'd throw that out there.

Also wrt this, a t-test makes the assumption of normality so you shouldn't be using t either.

[runthemover 39]

a lot of it depends on what exactly the assignment is. Does the professor just want you to calculate a z-score for the purpose of just going through the motions?when you see it's not normal, you should probably look at outliers and stratifying groups to 'solve' this problem for other data analysisthe z-score, other than telling us how far from the mean it is in standard deviations can tell us (for normal data) where a certain percentage of our data falls. so this is the only thing we really can't do with this set.it just seems quite silly to me that apparently the professor wants a z-score to explain that certain data is farther from the mean than other data

[vbnautilus 48]

What if you know the shape?

You don't. (You may know the shape of the sample, but that's not the same as knowing the underlying distribution from which it was sampled). Often it is safe to assume normality (due to the central limit theorem), but when you have evidence that your distribution is not normal, that assumption is not warranted and non-parametric stats are used. It's generally not safe to assume another specific distribution unless you have a pretty good a-priori reason to.

[SpiderGuard 0]

How dare you!SpiderGuard says he wants a Z-score.I told him he can have a Z-score without a Normal DistributionWhat are you disagreeing with?"Probability Values Associated with Z-scores" ... seems like you are thinking in terms of Normal, as my post above your noted.ps. Why are you disreguarding the other Parametric distributions. Just curious. Seems like something else could be fit; but alas, I don't recall the general shapes of these.

So I'm thinking z-scoring is not the way to go...I'm trying to come up with a hypothesis test for this problem, so I'll take any suggestions of places to look.(I took stats for political scientists...we're even below the psych folks) ;)Edit: It's not a homework assignment so there are no real guidelines. Just something that's accurate and tests that the distances are significantly different from each other.

[vbnautilus 48]

So I'm thinking z-scoring is not the way to go...I'm trying to come up with a hypothesis test for this problem, so I'll take any suggestions of places to look.(I took stats for political scientists...we're even below the psych folks) ;)Edit: It's not a homework assignment so there are no real guidelines. Just something that's accurate and tests that the distances are significantly different from each other.

Can you give an example of the type of hypothesis you are trying to test?

[Actuary 3]

I have nothing more significant to add.So far in my work I've had little opportunity to deal with what I learned about fitting distributions.I'm still not 100% clear on why a Z-score is not the best thing for a simple level analysis.

[SpiderGuard 0]

Can you give an example of the type of hypothesis you are trying to test?

Let me give a little background on what the plot means before I get into the hypothesis test so I can explain what I'm doing here. I'm certainly open to any suggestions on where I've gone wrong.So the X Axis represents distance between the target word and the word "American." Words with small distances are related, words with large distances are unrelated. So the basic hypothesis is that "we" and "American" are related words (have a small distance)The picture represents the distribution of distances between American and all other words in the corpus. My thoughts: There are a couple different ways of doing this. The first is a pseudo-Neyman Pearson hypothesis testing framework (i.e. "we" is far enough away from the mean that it can't be that far away just by chance). The second is maybe some sort of Monte Carlo simulation...But yeah - there are many people on this board who are much better than I am at the statistics so I'm open to any and all suggestions of why I'm wrong and better ways to do this. It's a fairly important part of my research agenda, so I'm interested in doing this right, not just doing it with the tools I currently have in my limited toolbox.Thanks so far to all who have participated, please keep it coming.

[SpiderGuard 0]

I have nothing more significant to add.So far in my work I've had little opportunity to deal with what I learned about fitting distributions.I'm still not 100% clear on why a Z-score is not the best thing for a simple level analysis.

It very well could be z-scoring. That's my initial thought, and I think it fits with what most political scientists would be comfortable with. I think I might need to come up with another distribution though...my problem is that I know enough to be dangerous with this stuff, but not enough to really be good.