Skip to content

Commit 668ba9d

Browse files
authored
Update README.md
Added phase 2 details and to-do list
1 parent 2e661a0 commit 668ba9d

File tree

1 file changed

+16
-0
lines changed

1 file changed

+16
-0
lines changed

docs/README.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,24 @@ This project connects to the OpenSEA stream (non-REST) API to steam NFT transact
1515
## Project Timeline
1616
### Phase 1 - Project Setup
1717
Setup of goals, basic project structure, timeline, virtual environments and version control.
18+
1. Kafka, Spark, Jupyter Docker images setup, docker-compose setup and 1st git commit.
19+
2. Node.js project directory setup and 1st git commit.
20+
3. Java Gradle project directory setup and 1st git commit.
21+
4. Azure account, git documentation creation and 1st git commit.
1822
### Phase 2 - Duct Tape / Rapid Prototyping Phase (Current Phase)
1923
Get all componented connected and the entire pipeline running in whatever form as quick as possible.
24+
1. Kafka-Spark-Jupyter Docker stack: run containers. (done)
25+
2. Kafka: Create test topics. (done)
26+
3. Spark: Create and run spark session. (done)
27+
4. Node.js app: Connect to OpenSEA API and produce json transaction console logs. (done)
28+
5. Node.js app: Produce live json transaction data to kafka topics. (done)
29+
6. Spark: Achieve consumption of kafka topic and basic transformation to another kafka topic. (done)
30+
7. Azure Lakehouse: Achieve creation of Bronze, Silver and Gold containers. (done)
31+
8. Azure Lakehouse: Achieve writing of simple delta file df data to lakehouse from local Spark Docker container. (done)
32+
To-do:
33+
1. Lakehouse-Tableau connection
34+
2. Lakehouse-Plotly Dash connection
35+
3. Kafka-Plotly Dash live connection
2036
### Phase 3 - Feature Complete Phase
2137
All main features implemented. Design Freeze.
2238
### Phase 4 - Stabilizing Phase

0 commit comments

Comments
 (0)