A pattern-based topic detection and analysis system on Chinese tweets

作者：Zhang, Lu^*; Wu, Zhiang; Bu, Zhan; Jiang, Ye; Cao, Jie

来源：Journal of Computational Science, 2018, 28: 369-381.

DOI：10.1016/j.jocs.2017.08.016

• Social media
• Topic detection
• Evolutionary analysis
• Sentimental analysis
• Weibo

Online social media is able to convey rich and timely information about real-world events. Uncovering events on social media and sensing topics from them can acquire much valuable information, which has attracted significant research effort. However, due to the large scale of data, to detect events or topics in real time is still a challenging problem. In this paper, we propose a Pattern-based Topic Detection and Analysis System (PTDAS) on Weibo, a Twitter-like platform in China. As one of the key components of the whole system, a FP-growth-like algorithm is employed to mine cosine interesting patterns from a set of tweets, and then summarize them as topics. Specially, in order to discover topics in real-time, we parallelize the algorithm on Spark for efficient mining. Along with pattern-based topic detection, we also present some analytic techniques, including both topic evolving analysis and sentimental analysis. Extensive experiments on the real-world data set demonstrate the effectiveness and efficiency of PTDAS.

• 出版日期2018-9
• 单位南京财经大学