Indian Journal of Science and Technology

Abstract

Objectives: Everyday millions of users are downloading files through internet. It usually takes more amount of time and often leads to internet traffic in order to satisfy various clients in distributed environment. At present, cloud provides vital storage solution for both enterprises and the individual users. Methods: The existing system uses dual direction file transfer technique to transfer a file from its either end of the direction in order to enhance the download speed. But this approach does not guarantee reliability and it also increases latency for file transfer if the servers are too far from each other. Moreover, this technique was invented for single client. If multiple servers want to download the same file it will increase the total time required for download. Findings: A novel file transfer tool using a message broker. Kafka has been proposed to transfer large sized file with enhanced reliability and security. Kafka is one of the latest message broker used by many enterprises. This technique reduces the time required for replicating the files on many servers and downloading into the target server. If there are multiple clients, Kafka makes use of its mirroring technique to make a copy of files on many servers and we can download it easily. Applications: File transfer using Kafka can be used by many companies for managing their log files and to transfer it quickly from one data center to another. It also ensures reliability and secure file transfer across the data centers in the world.
Keywords: Cloud Computing, Distributed System, Kafka, Message Broker, Reliability