3. Setting Up Connectors and Catalogs
One of Trino's core strengths is its ability to connect to and query various external data sources. This is achieved through Connectors and Catalogs.
3.1. Understanding Trino Connectors
A Connector is a plugin that enables communication between Trino and a specific data source (e.g., Hive, MySQL, PostgreSQL, Kafka, etc.). Each connector contains logic specific to its data source.
3.2. Creating Catalog Configuration Files
To connect to a new data source, you need to create a .properties file in the Trino configuration directory (typically /etc/trino/catalog). For example, to connect to a MySQL database, create a mysql.properties file:
# /etc/trino/catalog/mysql.properties
connector.name=mysql
connection-url=jdbc:mysql://your_mysql_host:3306/your_database
connection-user=your_username
connection-password=your_passwordAfter creating the file and restarting the Trino server, a new catalog named mysql will be available for use in the Trino UI or CLI. You can now execute queries like SELECT * FROM mysql.your_schema.your_table;.
3.3. Common Connector Types
- Hive: Integrates with Hadoop HDFS and Apache Hive Metastore for data lake querying.
- PostgreSQL/MySQL: For querying relational databases.
- Kafka: For querying real-time streaming data.
- Amazon S3: For querying data stored in S3.
- Memory: An in-memory catalog for development and testing.