Studies have shown the potential of social media as a valuable source of real-time information for disaster monitoring through machine learning. However, the scarcity of labeled data used for training, especially during the onset of a disaster event, leads to poor machine learning output. This study introduced a method of improving the performance of machine learning classifiers by developing proxy labels for unlabeled datasets to increase the amount of training data. The design framework applied different concepts in machine learning including semi-supervised learning, Siamese neural network and transfer learning. The resulting models, together with traditional and deep neural network classifiers, were subjected to a comparative analysis experiment using disaster-related tweets collected within the Philippines as benchmark datasets. Results showed that the proposed method produced better F1-scores as compared to traditional machine learning and deep neural network approaches. Integration of this method to disaster monitoring systems can drastically lessen the need for a large workforce to provide manual labelling on collected social media data, thus, saving on cost, time and resources.
Keywords: Disaster monitoring on social media, machine learning techniques for text classification, transfer learning, word embeddings[1] Nguyen, T.D., Joty, S.R., Imran, M., Sajjad, H., and Mitra, P. (2016). Applications of Online Deep Learning for Crisis Response Using Social Media Information. In Proceedings of the 4th International Workshop on Social Web for Disaster Management (SWDM’16), 4. Retrieved from https://arxiv.org/pdf/1610.01030.pdf.
[2] Imran, M., Castillo, C., Lucas, J., Meier, P., and Vieweg, S. (2014). AIDR: Artificial Intelligence for Disaster Response. In Proceedings of the 23rd International Conference on World Wide Web, 159-162. Retrieved from https://doi.org/10.1145/2567948.2577034.
[3] Patel, F., and Soni, N. (2012). Text mining: A Brief survey. International Journal of Advanced Computer Research, 2(6), 243-248. Retrieved from https://pdfs.semanticscholar.org/11c4/6d00a0e136e8e4e27aa15fbb8c9111cdee75.pdf.
[4] Qadir, J., Ali, A., Rasool, R. U., Zwitter, A., Sathiaseelan, A., and Crowcroft, J. (2016). Crisis Analytics: Big Data Driven Crisis Response. Journal of International Humanitarian Action, 1(1), 1-21. Retrieved from https://doi.org/10.1186/s41018-016-0013-9
[5] Murphy, K. (2012). Machine Learning: A Probabilistic Perspective. Cambridge, Massachusetts: The Massachusetts Institute of Technology (MIT) Press. ISBN: 9780262018029.
[6] Russell, S., and Norvig, P. (2009). Artificial Intelligence: A Modern Approach (3rd ed.). Upper Saddle River, New Jersey: Prentice Hall Press. ISBN: 9780136042594.
[7] Alam, F., Joty, S.R., and Imran, M. (2018). Graph Based Semi-supervised Learning with Convolution Neural Networks to Classify Crisis Related Tweets. In Proceedings of the 12th International AAAI Conference on Web and Social Media, 12, 556-559. Retrieved from https://arxiv.org/pdf/1805.06289v1.pdf.
[8] Chapelle, O., Scholkopf, B., and Zien, A., Eds. (2006). Semi-Supervised Learning. Cambridge, Massachusetts: The Massachusetts Institute of Technology (MIT) Press. ISBN: 9780262033589.
[9] Pan, S. J., and Yang, Q. A. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359. Retrieved from https://doi.org/10.1109/TKDE.2009.191.
[10] Neculoiu, P., Versteegh, M. and Rotaru, M. (2016). Learning Text Similarity with Siamese Recurrent Networks. In Proceedings of the 1st Workshop on Representation Learning for NLP, 148-157. Retrieved from https://doi.org/10.18653/v1/W16-1617.
[11] Olteanu, A., Castillo, C., and Vieweg, S. (2015). What to Expect When the Unexpected Happens: Social Media Communications Across Crises. In Proceedings of the ACM 2015 Conference on Computer Supported Cooperative Work and Social Computing (CSCW ‘15), 994-1009. Vancouver, BC, Canada: ACM. Retrieved from https://doi.org/10.1145/2675133.2675242.
[12] Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems, 2, 3111-3119. Retrieved from https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf.
[13] Barbieri, F., Ronzano, F., and Saggion, H. (2016). What does this Emoji Mean? A Vector Space Skip-Gram Model for Twitter Emojis. In Proceedings of the 10th International Conference on Language Resources and Evaluation, 10, 3967-3972. Retrieved from https://www.aclweb.org/anthology/L16-1626.pdf.
[14] Dietterich, T.G. (1998) Approximate statistical test for comparing supervised classification learning algorithms. Neural Computation. Neural Computation, 10(7), 1895–1923. Retrieved from https://doi.org/10.1162/089976698300017197.
[15] Sasaki, Y. (2007). The truth of the F-measure. Teach Tutor Mater, 1(5), 1-5. Retrieved from https://www.researchgate.net/publication/268185911_The_truth_of_the_F-measure.
[16] Hollander, M., and Wolfe, D. (2013). Nonparametric Statistical Methods (3rd ed.). New York: Wiley Interscience Publication. ISBN: 9781118553299.