我想基于遵循 Zipf 分布的單詞(來自字典)創建數據源(用 Java 編寫)。所以我來到了Apache commons 庫的ZipfDistribution和NormalDistribution 。不幸的是,有關如何使用這些類的信息很少。我嘗試做一些測試,但我不確定我是否以正確的方式使用它。我僅遵循每個構造函數的文檔中所寫的內容。但結果似乎并不“分布均勻”。import org.apache.commons.math3.distribution.NormalDistribution;import org.apache.commons.math3.distribution.ZipfDistribution;import java.io.BufferedReader;import java.io.IOException;import java.io.InputStream;import java.io.InputStreamReader;import java.net.URL;public class ZipfDistributionDataSource extends RichSourceFunction<String> { private static final String DISTINCT_WORDS_URL = "https://raw.githubusercontent.com/dwyl/english-words/master/words_alpha.txt"; public static void main(String[] args) throws Exception { ZipfDistributionDataSource zipfDistributionDataSource = new ZipfDistributionDataSource(); StringBuffer stringBuffer = new StringBuffer(zipfDistributionDataSource.readDataFromResource()); String[] words = stringBuffer.toString().split("\n"); System.out.println("size: " + words.length); System.out.println("Normal Distribution"); NormalDistribution normalDistribution = new NormalDistribution(words.length / 2, 1); for (int i = 0; i < 10; i++) { int sample = (int) normalDistribution.sample(); System.out.print("sample[" + sample + "]: "); System.out.println(words[sample]); } System.out.println(); System.out.println("Zipf Distribution"); ZipfDistribution zipfDistribution = new ZipfDistribution(words.length - 1, 1); for (int i = 0; i < 10; i++) { int sample = zipfDistribution.sample(); System.out.print("sample[" + sample + "]: "); System.out.println(words[sample]); } }
1 回答

青春有我
TA貢獻1784條經驗 獲得超8個贊
從代碼的角度來看,您使用它很好:) 問題在于假設源材料是按 Zipf 排序的,而它顯然是按字母順序排列的。使用的全部意義ZipfDistribution
在于,words[0] 必須是最常見的單詞(提示:它是“the”),并且大約是words[1] 頻率的兩倍)等。
添加回答
舉報
0/150
提交
取消