Enter Postgres's container: docker-compose exec postgres psql -h localhost -U postgresĢ. Order_status BOOLEAN NOT NULL - Whether order has been placed Order_id INTEGER NOT NULL AUTO_INCREMENT PRIMARY KEY, (default,"spare tire","24 inch spare tire") (default,"jacket","water resistent black wind breaker"), (default,"rocks","box of assorted rocks"), (default,"hammer","16oz carpenter's hammer"), (default,"hammer","14oz carpenter's hammer"), (default,"hammer","12oz carpenter's hammer"), (default,"12-pack drill bits","12-pack of drill bits with sizes ranging from #40 to #3"), (default,"car battery","12V car battery"), VALUES (default,"scooter","Small 2-wheel scooter"), Id INTEGER NOT NULL AUTO_INCREMENT PRIMARY KEY,ĪLTER TABLE products AUTO_INCREMENT = 101 Enter MySQL's container: docker-compose exec mysql mysql -uroot -p123456Ģ. Preparing Data in Databases Preparing Data in MySQLġ. Download the following required JAR package and put them under flink-1.13.2/lib/:ĭownload links are only available for stable releases.Download Flink 1.13.2 and unzip it to the directory flink-1.13.2.Preparing the Required Flink and JAR Package We can also visit to see if Kibana is running normally. Run docker ps to check whether these containers are running properly. This command automatically starts all the containers defined in the Docker Compose configuration in a detached mode. Run the following command in the directory that contains the docker-compose.yml file to start all the containers: docker-compose up –d Kibana is used to visualize the data in Elasticsearch.Elasticsearch is mainly used as a data sink to store enriched orders.Postgres: The shipments table will be stored in the database.MySQL: The products and orders tables will be stored in the database and joined with data in Postgres to enrich the orders.The Docker Compose environment consists of the following containers: The components required in this demo are all managed in containers, so we will use docker-compose to start them.Ĭreate a docker-compose.yml file using the following content: version: '2.1' Prepare a Linux or MacOS computer with Docker installed. The overview of the architecture is listed below: The entire process uses standard SQL syntax without a single line of Java/Scala code or IDE installation. We want to enrich the orders using the product and shipment table and load the enriched orders to Elasticsearch in real-time.Īll exercises in this tutorial are performed in the Flink SQL CLI. The product and order data stored are in MySQL, and the shipment data related to the order are stored in Postgres. Let's imagine we are running an e-commerce business. The demos of this tutorial are based on the Docker environment and will be performed in the Flink SQL CLI, which only involves SQL, without a single line of Java/Scala code and without installing an IDE.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |