Differences

This shows you the differences between two versions of the page.

--- dido:public:s_cli:05_contents:03_prt:08_basic_dido_objects:start [2021/06/14 16:18]
nick [A.1.2 Details of Ethereum Blockchain Objects]
+++ dido:public:s_cli:05_contents:03_prt:08_basic_dido_objects:start [2022/02/03 15:00] (current)
157.90.182.28 ↷ Links adapted because of a move operation
@@ Line 2: / Line 2: @@
 [[dido:public:s_cli:05_contents:01_prt:02_basics:start| Return to DIDO CLI Background]]
-===== A.1.2 The Ethereum Blockchain Data Flow =====
+===== A.1.1 The Ethereum Blockchain Data Flow =====
 [[dido:public:s_cli:05_contents:03_prt:08_basic_dido_objects:start | Return to Top]]
-Ethereum, as well as most DIDO Platforms, use a [[dido:public:ra:xapend:xapend.a_glossary:r:rest]] model as the basis of their [[dido:public:ra:xapend:xapend.a_glossary:a:api]]. [[dido:public:ra:xapend:xapend.a_glossary:e:ethereum_node]] (i.e.,Clients) formulate a request in [[dido:public:ra:xapend:xapend.a_glossary:x:xml]] or [[dido:public:ra:xapend:xapend.a_glossary:j:json]] and send it to a RESTful API service that processes the request and calls the [[dido:public:ra:xapend:xapend.a_glossary:e:evm]] in the [[dido:public:ra:xapend:xapend.a_glossary:e:ethereum_node]] using a [[dido:public:ra:xapend:xapend.a_glossary:r:rpc]]. The EVM processes the request and returns any required data back to the API Service as part of the RPC. The API Service then creates an appropriate XML or JSON response and returns it to the client over [[dido:public:ra:xapend:xapend.a_glossary:h:http]] or [[dido:public:ra:xapend:xapend.a_glossary:h:https]].
+[[dido:public:ra:xapend:xapend.a_glossary:e:ethereum|Ethereum]], as well as most DIDO Platforms, use a [[dido:public:ra:xapend:xapend.a_glossary:r:rest]] model as the basis of their [[dido:public:ra:xapend:xapend.a_glossary:a:api]]. [[dido:public:ra:xapend:xapend.a_glossary:e:ethereum_node]] (i.e.,[[dido:public:ra:xapend:xapend.a_glossary:c:client|Clients]]) formulate a request in [[dido:public:ra:xapend:xapend.a_glossary:x:xml]] or [[dido:public:ra:xapend:xapend.a_glossary:j:json]] and send it to a [[dido:public:ra:xapend:xapend.a_glossary:r:restful|RESTful API]] service that processes the request and calls the [[dido:public:ra:xapend:xapend.a_glossary:e:evm]] in the [[dido:public:ra:xapend:xapend.a_glossary:e:ethereum_node]] using a [[dido:public:ra:xapend:xapend.a_glossary:r:rpc]]. The EVM processes the request and returns any required data back to the API Service as part of the RPC. The API Service then creates an appropriate XML or JSON response and returns it to the client over [[dido:public:ra:xapend:xapend.a_glossary:h:http]] or [[dido:public:ra:xapend:xapend.a_glossary:h:https]].
 <figure>
 {{  :dido:public:s_cli:05_contents:03_prt:08_basic_dido_objects:screen_shot_2021-06-12_at_4.07.17_pm.png?600  |}}
-<caption>The Flow of Data from Clients to Ethereum Blockchain</caption>
+<caption>The Flow of Data from Clients to Ethereum [[dido:public:ra:xapend:xapend.a_glossary:b:blockchain|Blockchain]]</caption>
 </figure>
@@ Line 36: / Line 36: @@
 </figure>
-  * <WRAP>//Keep in mind, Ethereum is the database, smart contracts are the data tables, and transactions from wallets are the rows in each table.//((
+  * <WRAP>//Keep in mind, Ethereum is the database, [[dido:public:ra:xapend:xapend.a_glossary:s:smart_contract|smart contracts]] are the data tables, and transactions from wallets are the rows in each table.//((
 Andrew Hong,
 Hands-On Tutorials,
@@ Line 46: / Line 46: @@
 </WRAP>
-===== A.1.2 Details of Ethereum Blockchain Objects =====
+===== A.1.3 Details of Ethereum Blockchain Objects =====
+[[dido:public:s_cli:05_contents:03_prt:08_basic_dido_objects:start | Return to Top]]
 Ethereum uses a [[dido:public:ra:xapend:xapend.a_glossary:d:dom]] [[dido:public:ra:xapend:xapend.a_glossary:d:dm]] rather than a [[dido:public:ra:xapend:xapend.a_glossary:r:relational_model]], however, the documents are shallow((
@@ Line 52: / Line 54: @@
 )) and lend themselves to mapping to the relational model well.
-The use of Google's BigQuery was designed for analyzing data on the order of billions of rows, using a SQL-like syntax. It runs on the Google Cloud Storage infrastructure and can be accessed with a REST-oriented application program interface (API).
+The use of Google's [[dido:public:ra:xapend:xapend.a_glossary:b:big_query|BigQuery]] was designed for analyzing data on the order of billions of rows, using a [[dido:public:ra:xapend:xapend.a_glossary:s:sql]]-like [[dido:public:ra:xapend:xapend.a_glossary:s:syntax|syntax]]. It runs on the Google Cloud Storage infrastructure and can be accessed with a REST-oriented application program interface (API).
-Trying to understand and use [[dido:public:ra:xapend:xapend.a_glossary:b:big_query]] developers have been able to tab into the Ethereum Distributed Ledger Technology (DLT) (i.e., blockchain) data using very close to standard SQL((
+Trying to understand and use [[dido:public:ra:xapend:xapend.a_glossary:b:big_query]] developers have been able to tab into the Ethereum [[dido:public:ra:xapend:xapend.a_glossary:d:dlt]] (i.e., [[dido:public:ra:xapend:xapend.a_glossary:b:blockchain]]) data using very close to standard SQL((
 Rif Kiamil,
 February 2021,
@@ Line 62: / Line 64: @@
 )).
-  - **Ehtereum Extract, Load, Transform** or **Ethereum ETL** converts Ethereum blockchain data into convenient formats such as [[dido:public:ra:xapend:xapend.a_glossary:c:csv]] files and relational databases.
+Figure {{ref>EtlArch}} provides a high-level flow of converting Ethereum Blockchain data into Google's BigQuery and as a consequence accessible using SQL. In this workflow, an Ethereum Node is deployed within a [[dido:public:ra:xapend:xapend.a_glossary:c:container|container]] orchestrated by a Kubernetes Engine. there are two paths through the data flow: a Real-Time path and a Daily Path.
-<figure>
+  * In the Daily Path, a [[dido:public:ra:xapend:xapend.a_glossary:s:snapshot|snapshot]] of the blockchain data is exported once a day, converted to [[dido:public:ra:xapend:xapend.a_glossary:c:csv]]  files, and then loaded into BigQuery where the data can be accessed using the BigQuery Console.
+  * In the Real-Time Path that has a lagtime delay to prevent orphaned blocks, the data is streamed to a publish/subscribe utility that can create either
+    * An ETL Dataflow is loaded into BigQuery where data can be accessed using the BigQuery Console
+    * An [[dido:public:ra:xapend:xapend.a_glossary:a:application|Application]] subscribes directly to a topic
+<figure EtlArch>
 {{  :dido:public:s_cli:05_contents:03_prt:08_basic_dido_objects:screen_shot_2021-06-14_at_9.51.14_am.png?700  |}}
 <caption>Blockchain ETL Architecture((
@@ Line 74: / Line 81: @@
 ))</caption>
 </figure>
+===== A.1.4 BigQuery Data Objects =====
+[[dido:public:s_cli:05_contents:03_prt:08_basic_dido_objects:start | Return to Top]]
+The following is a list of Ethereum Data Objects (i.e., tables) that are available using [[dido:public:ra:xapend:xapend.a_glossary:b:big_query]] using [[dido:public:ra:xapend:xapend.a_glossary:s:sql]].
 <nspages -tree -r -exclude -subns -pagesInNs -h1 -textNs="">

DIDO Wiki

User Tools

Site Tools

Differences

Page Tools